[katello-devel] CLI tests with better granularity

[Dmitri Dolguikh]

On 11-07-25 11:26 AM, Lukas Zapletal wrote:
> On 07/25/2011 02:12 PM, Dmitri Dolguikh wrote:
>> I'm not suggesting getting rid of cli tests, but rather implementing
>> them as a set of (mostly) unit tests, plus a set of katello api tests
>> (which can be run once-twice a day). Current cli tests are quite slow
>> and fragile, and they will only get more slow and fragile with time.
>
> Hmm, let's don't call it unit tests. I would suggest "rspec examples" 
> or "rspec tests" here. I was confused, because:
>
> Unit test is a method by which individual units of source code are 
> tested to determine if they are fit for use. A unit is the smallest 
> testable part of an application. -- wikipedia
>
> Frankly I would rather focus on improving code coverage than 
> re-writing some cli tests as rspec tests. Maybe the most important ones. 
Our current cli tests provide a great coverage - they are end-to-end 
tests. The problem is their fragility, how useful they are for isolating 
bugs (not very), etc. I would suggest keeping cli tests around for now, 
but start adding unit- or whatever else you'd like to call these tests. 
The point is that they should test parts of cli in isolation from each 
other (to me that is most important aspect of unit tests).

> Speaking about slowness there's room for improvement in our rspec 
> suite as well ;-)
I think Bryan added stories to deal with slow tests (and a couple of 
other issues).

-d
>
> AFAIK Garik is already working on recurrent running of CLI tests.
>