[katello-devel] Summary: Katello slow tests

[David Davis]

Dimitri,

Currently Mike has a change that he is working on adding to master that incorporates the parallel_tests gem into the code. Using that, he was able to bring the total test time from 12-14 minutes to 2 minutes in the test/jenkins environment and to about 4 minutes on his local environment.  

Also, Eric and I have added FactoryGirl to the code on our pulpv2 branch. I'll probably give a demo of how to use it to speed up tests at the next Deep Dive. 

Your other ideas sound good too. Let me know if you want to coordinate something as I could help modifying the existing tests to access the database less. From previous experience, IO operations during testing tend to be the bottleneck.

Thanks.

David

----- Original Message -----
> From: "Dmitri Dolguikh" <dmitri at redhat.com>
> To: "katello-devel" <katello-devel at redhat.com>
> Sent: Friday, October 12, 2012 11:45:29 AM
> Subject: [katello-devel] Summary: Katello slow tests
> 
> 
> A spent a couple of days in total looking at our developer tests and
> their performance (I managed to get ruby-prof to profile a few test
> suites, which was a great help. I have the files for those who are
> interested).
> 
> 
> Some stats: we have 2280 or so tests. The total execution time of all
> specs on my machine is about 14 minutes. About two thirds of tests
> are slower than 0.1 sec. Majority, if not all tests in
> spec/controllers access the db. In some test suites time spent on db
> operations amounts to high 40-ties of total test run time. Tests in
> spec/models have no clear separation between model and glue-layer
> tests, besides context changes.
> 
> 
> Some areas of improvement:
> - Turn controller tests into real unit tests - isolate them from the
> db. This will be easier if something like Factory-Girl [1] was used
> for object graph generation.
> - we have a few tests that exercise views, perhaps they should be
> moved out into a separate module, as they can be quite slow.
> - Break up model tests into:
> - tests that access db (test queries, etc)
> - tests that exercise orchestration logic. These will effectively be
> unit-tests.
> 
> 
> Bryan: I think the stories can be broken by type of tests, either one
> story per test-suite (ie controller tests, model tests), with work
> broken up by model/controller. Alternatively, we could have one
> story per test (ie modify systems_controller_spec, etc). Please let
> me which is more convenient for you.
> 
> 
> Thoughts, opinions?
> -d
> 
> 
> 
> [1] https://github.com/thoughtbot/factory_girl
> 
> 
> _______________________________________________
> katello-devel mailing list
> katello-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/katello-devel