[katello-devel] Summary: Katello slow tests

[Dmitri Dolguikh]

A spent a couple of days in total looking at our developer tests and 
their performance (I managed to get ruby-prof to profile a few test 
suites, which was a great help. I have the files for those who are 
interested).


Some stats: we have 2280 or so tests. The total execution time of all 
specs on my machine is about 14 minutes. About two thirds of tests are 
slower than 0.1 sec. Majority, if not all tests in spec/controllers 
access the db. In some test suites time spent on db operations amounts 
to high 40-ties of total test run time. Tests in spec/models have no 
clear separation between model and glue-layer tests, besides context 
changes.


Some areas of improvement:
  - Turn controller tests into real unit tests - isolate them from the 
db. This will be easier if something like Factory-Girl [1] was used
  for object graph generation.
   - we have a few tests that exercise views, perhaps they should be 
moved out into a separate module, as they can be quite slow.
  - Break up model tests into:
   - tests that access db (test queries, etc)
   - tests that exercise orchestration logic. These will effectively be 
unit-tests.


Bryan: I think the stories can be broken by type of tests, either one 
story per test-suite (ie controller tests, model tests), with work 
broken up by model/controller. Alternatively, we could have one story 
per test (ie modify systems_controller_spec, etc). Please let me which 
is more convenient for you.


Thoughts, opinions?
-d



[1] https://github.com/thoughtbot/factory_girl

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/katello-devel/attachments/20121012/afd58cf2/attachment.htm>