Regression testing

Gregory Maxwell gmaxwell at gmail.com
Fri Mar 18 20:09:23 UTC 2005


A number of pieces of software included in Fedora have automated
regression tests included with their source.  I think it would be
handy if there was a way to easily install and run these tests in a
uniform fashion.

One way to accomplish this would be to have the spec optionally build
a packagename-regression rpm, which includes the scripts and data
needed to perform the automated regression testing.

During testing users could choose to install all the available
regression rpms, and run the tests.  Even a fairly limited set of
tests would likely do a good job at finding problems due to lower
level infrastructure changes (gcc bugs, glibc, kernel, etc).

Furthermore, distribution wide uniform regression testing would
encourage the creation of regression test suites for all packages...
Right now, no one will likely run a regression suite for 'bc', so no
one bothers writing one... but it would get run as part of a
distribution wide testing.  If all the silly little apps had some
basic testing we would be much more likely to catch bugs in the system
as a whole.

It would also be useful to track the performance of these tests, so
that performance regressions on various platforms could be spotted and
addressed early on, and not just reactively.

Am I totally off the mark here?  Has work been done on this front already?




More information about the fedora-devel-list mailing list