F11 Xorg/Mesa/DRI testing proposal

"Jóhann B. Guðmundsson" johannbg at hi.is
Fri Feb 13 09:27:17 UTC 2009


FD Cami wrote:
> Hi,
>
> We would like to organize efficient Xorg/Mesa/DRI testing before F11 GA.
>
> Our goal is to record what works as expected on mainstream, supported
> hardware with free drivers (nv, i810, radeon, soon nouveau), what does
> not, and track regressions from Alpha to GA.
>
>   

True

> We can't rely on beta-testers alone to find the critical bugs in Xorg
> before the GA. However, the Fedora project has a large number of
> contributors, probably covering most available mainstream hardware ;
> there is currently no way to tap that existing resource, because we
> don't know who owns which hardware.
>
>   

True

> Each of us having limited ressources both in time and hardware at our
> disposal, we'd like to :
>
> 0. be able to use a opt-in searchable database of hardware belonging to
>    FAS account holders ; linking FAS accounts and smolt profiles, at the
>    account holder's discretion has been suggested and would work well.
>    We may also need to use that database to contact contributors that
>    would also have elected to participate in the testing program.
>
>   

I discussed this with Mike McGratg a while back after me and John<some 
number)
and been tossing ideas back and fourth on fedora-qa. I'm going to 
forward our
conversation to the list so we can share our ideas on smolt improvements 
in a new thread.
See thread [Fwd: Re: Bringing smolt to the next level..]
> 1. be able to write and use test plans efficiently. The wiki is suitable
>    for that task.
>
>   
Well suitable is a strong word hopefully will be replaced by something 
better sooner rather than later :)
> 2. record the results of test runs ; those should be at least searchable
>    per application, per driver and per actual hardware ID. Any bugs
>    encountered will of course be recorded in Bugzilla, but we also need
>    to know which tests were successfully completed. A few solutions have
>    already been suggested, including the laptop.org wiki scripts solution
>    (see http://wiki.laptop.org/go/Test_cases_8.2.1 for instance), or a
>    dedicated application like TestLink or Nitrate (which may not be ready
>    for F11).
>
>   
Afaik nitrate wont be ready in F11 time frame and should only be thought 
to enter F12
Cycle.
> 3. regularly schedule test sessions, using the opt-in system mentioned at
>    step #0.
>
>   
Yup.
> We will need some help from -infra for at least #0 and #2.
>
>   
Yup
> Running test cases only requires having the missing hardware, some free
> time, and being able to use bugzilla.
> Possible test cases are basic 2D tests, for instance x11perf and full
> screen movies ; 3D tests, like glxgears and mesa-demos, then GL
> regression testing with using piglit.
> Desktop effects using Compiz/KDE would be next, then to 3D applications
> like Blender, ending with 3D games like GL-117 and FlightGear for
> instance.
>
>   
Start writing test cases that's the first thing that needs to be done.
> Obviously, this methodology could be expanded to other components of
> Fedora as well. We are specifically thinking of testing for regressions
> in the audio subsystem and validating updates in Bodhi.
>   
JBG
-------------- next part --------------
A non-text attachment was scrubbed...
Name: johannbg.vcf
Type: text/x-vcard
Size: 372 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/fedora-test-list/attachments/20090213/f521a4df/attachment.vcf>


More information about the fedora-test-list mailing list