[katello-devel] Proposal - Better Continuous Integration

Tue Nov 20 21:48:38 UTC 2012

David Davis <daviddavis at redhat.com> writes:

So, in order to fix the build, you don't necessarily have to fix your
*own* violation, you just have to fix *someone's*?  I'm not sure if this
is a good or bad thing, but it's interesting. :)

-jeff

> No, cane is more like pylint or jshint in that it'll actually return
> an exit code of 1 if a certain number of violations is exceeded. So
> currently we have like 1580 and we could cap it there so if any more
> violations get added, the CI job will fail (just like pylint).
>
> David
>
> ----- Original Message -----
>> From: "Lukas Zapletal" <lzap at redhat.com>
>> To: katello-devel at redhat.com
>> Sent: Tuesday, November 20, 2012 2:44:40 PM
>> Subject: Re: [katello-devel] Proposal - Better Continuous Integration
>> 
>> I am under impression that cane and abcmetric are some kind of metric
>> tools which can told you how complex is it. It generates statistics
>> and
>> I bet they have not been built for integration with CI tools which
>> just
>> say Green or Red.
>> 
>> Our experiences from the past is the statistic just gets generated -
>> and
>> that's it. We need to talk about them.
>> 
>> LZ
>> 
>> On Tue, Nov 20, 2012 at 01:39:03PM -0500, David Davis wrote:
>> > So I think people will most definitely check the output and analyze
>> > it when their pull requests don't get merged because Travis is
>> > failing. [1]
>> > 
>> > [1] Styling conventions (like lines < 120 character and no
>> > whitespace) can be overridden using exclusion rules in cane.
>> > 
>> > ----- Original Message -----
>> > > From: "Lukas Zapletal" <lzap at redhat.com>
>> > > To: katello-devel at redhat.com
>> > > Sent: Tuesday, November 20, 2012 12:54:42 PM
>> > > Subject: Re: [katello-devel] Proposal - Better Continuous
>> > > Integration
>> > > 
>> > > This is not a problem indeed. The issue is we need to check the
>> > > output
>> > > regularly and analyze it... ;-)
>> > > 
>> > > LZ
>> > > 
>> > > On Tue, Nov 20, 2012 at 09:31:55AM -0500, David Davis wrote:
>> > > > So I've added cane [1] to our project [2] which is like pylint
>> > > > for
>> > > > Ruby. Right now it's set up to check the ABC complexity of each
>> > > > function [3] and also it checks style (line length (120 chars),
>> > > > trailing white space, etc). I'm not too sure about using the
>> > > > ABC
>> > > > metric but the style metrics seems excellent.
>> > > > 
>> > > > Also, cane is set to fail if we get above a certain threshold.
>> > > > It
>> > > > only takes about 10-20 seconds to scan the entire project so I
>> > > > really don't see a reason not to run it against each pull
>> > > > request.
>> > > > 
>> > > > [1] https://github.com/square/cane
>> > > > [2] https://github.com/Katello/katello/pull/1086
>> > > > [3] http://c2.com/cgi/wiki?AbcMetric
>> > > > 
>> > > > David
>> > > > 
>> > > > ----- Original Message -----
>> > > > > From: "Dmitri Dolguikh" <dmitri at redhat.com>
>> > > > > To: katello-devel at redhat.com
>> > > > > Sent: Tuesday, November 20, 2012 9:20:42 AM
>> > > > > Subject: Re: [katello-devel] Proposal - Better Continuous
>> > > > > Integration
>> > > > > 
>> > > > > On 20/11/12 01:25 PM, Petr Chalupa wrote:
>> > > > > > I think we should spend some time on getting better
>> > > > > > continuous
>> > > > > > integration. Travis is great but I think we need more than
>> > > > > > just
>> > > > > > Unit
>> > > > > > tests.
>> > > > > >
>> > > > > > What I would like to see for each pull-request to be run
>> > > > > > - all unit tests
>> > > > > > - building of rpms
>> > > > > > - running katello-configure
>> > > > > > - running system-tests (smoke tests)
>> > > > > >
>> > > > > > It would be also nice to have other thinks checked:
>> > > > > > - number of chars per line
>> > > > > > - pylint
>> > > > > > - afaik there is some script to check gettext strings
>> > > > > >
>> > > > > > ## Benefits
>> > > > > >
>> > > > > > - a developer doesn't have to run these tests locally
>> > > > > > (which
>> > > > > > takes
>> > > > > > time) until he is notified that something actually broke
>> > > > > > - master will be much more stable, bugs are solved in
>> > > > > > pull-requests
>> > > > > > by
>> > > > > > theirs authors (faster), also bug finding scope is given by
>> > > > > > introduced
>> > > > > > changes (because master is fine)
>> > > > > > - more stable nightly because master is stable
>> > > > > > - less time spend on testing, bug-finding and
>> > > > > > nightly-fixing by
>> > > > > > developers
>> > > > > >
>> > > > > > I think we should pay more attention to this and add story
>> > > > > > for
>> > > > > > next
>> > > > > > sprint to get it fixed.
>> > > > > >
>> > > > > > "As a developer I would really like to have CI for each
>> > > > > > pull-request."
>> > > > > >
>> > > > > > Petr
>> > > > > >
>> > > > > > _______________________________________________
>> > > > > > katello-devel mailing list
>> > > > > > katello-devel at redhat.com
>> > > > > > https://www.redhat.com/mailman/listinfo/katello-devel
>> > > > > 
>> > > > > If we can pull off running integration tests on every pull
>> > > > > request, I
>> > > > > think we should do it; I have my doubts this is feasible
>> > > > > however.
>> > > > > 30
>> > > > > mins per *each* pull request (and the time is only going to
>> > > > > go
>> > > > > up) is
>> > > > > quite a bit. The value of these tests is somewhat limited
>> > > > > too, as
>> > > > > they
>> > > > > are not truly integration tests, since each set of changes is
>> > > > > tested
>> > > > > independently from changes in other pull requests.
>> > > > > 
>> > > > > I don't think it's worth building an rpm for each pull
>> > > > > request
>> > > > > either.
>> > > > > For which platform? would koji people even support such a use
>> > > > > of
>> > > > > infrastructure?
>> > > > > 
>> > > > > -d
>> > > > > 
>> > > > > _______________________________________________
>> > > > > katello-devel mailing list
>> > > > > katello-devel at redhat.com
>> > > > > https://www.redhat.com/mailman/listinfo/katello-devel
>> > > > > 
>> > > > 
>> > > > _______________________________________________
>> > > > katello-devel mailing list
>> > > > katello-devel at redhat.com
>> > > > https://www.redhat.com/mailman/listinfo/katello-devel
>> > > 
>> > > --
>> > > Later,
>> > > 
>> > >  Lukas "lzap" Zapletal
>> > >  #katello #systemengine
>> > > 
>> > > _______________________________________________
>> > > katello-devel mailing list
>> > > katello-devel at redhat.com
>> > > https://www.redhat.com/mailman/listinfo/katello-devel
>> > > 
>> 
>> --
>> Later,
>> 
>>  Lukas "lzap" Zapletal
>>  #katello #systemengine
>> 
>> _______________________________________________
>> katello-devel mailing list
>> katello-devel at redhat.com
>> https://www.redhat.com/mailman/listinfo/katello-devel
>> 
>
> _______________________________________________
> katello-devel mailing list
> katello-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/katello-devel

-- 
Jeff Weiss
Principal Quality Assurance Engineer
jweiss at redhat.com
(919)886-6533