[katello-devel] Rspec and/or mini-test?

[Justin Sherrill]

On 03/09/2013 08:43 AM, Bryan Kearney wrote:
> On 03/08/2013 10:54 AM, Jordan OMara wrote:
>> On 08/03/13 10:44 -0500, Brad Buckingham wrote:
>>>
>>> As for all of the existing rspec tests, my 2cents is,
>>> - if a test requires updates (e.g. as app changes), it could be
>>> removed from rspec and added to minitest
>>> - if a developer wants to 'on the side' rewrite tests in minitest,
>>> that's ok, but I wouldn't make that a focus given we have a lot of
>>> features that should get priority
>>>
>>
>> +1
>>
>>
>> _______________________________________________
>> katello-devel mailing list
>> katello-devel at redhat.com
>> https://www.redhat.com/mailman/listinfo/katello-devel
>>
> Why are we adding a new testing framework? Is it fixing a problem in 
> the current frameworks?

So as part of the pulpv2 we tried to tackle improving our testing, some 
of the issues we desired to tackle:

- Our tests are written very poorly written and are extremely slow
   * We manually create()  many many objects before every test, in a lot 
of cases we are creating the org, environment, provider, product, etc... 
before every test.  This slows down the tests dramatically.
   * Over the past 2 years that 'model' has not really changed, everyone 
keeps plugging along in the existing rspec framework we have setup, and 
things only get worse.
- Working with the glue layer was a nightmare
   * In many places we stubbed glue layer calls which seems simple on 
the surface but due to the complex nature of our interactions and the 
responses these backend systems would give, it was not fun
   * No way to test glue layer functions against backend services
- The above could not be easily rectified without re-writing the spec tests
- In general we had heard a LOT of complaining about rspec.  Most of the 
tests were written as unit tests which is not the intent, and the syntax 
can be confusing.  After working with it for over a year, most of the 
people we talked to unofficially did not seem to enjoy it.

This was all covered in a deep dive last fall IIRC (eric lead it)

So what does minitest (and our work to use minitest) give us?

- Easier testing of models with the glue layer disabled (Dont' need a 
backend engine, dont' use it!)
- VCR testing of backend engines
- Easier to read tests
- Easier to write test

Could we have gotten the above with rspec?  Probably, but since the 
'framework' we had built around the tests needed to be torn down 
anyways, it made more sense to start completely fresh.  And since people 
seemed to prefer the unit style of tests, we went with that.


-Justin

>
> -- bk
>
> _______________________________________________
> katello-devel mailing list
> katello-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/katello-devel