We can't target any user without fixing these 5 things

[William Jon McCann]

Hey Chris,

On Thu, Oct 8, 2009 at 4:31 PM, Christopher Aillon <caillon at redhat.com> wrote:
> (hm, apparently my VPN connection died on the plane when I thought I had
> sent this last Saturday...)
>
> I think perhaps the single largest issue we have is a lack of
> reliability/predictability during a Fedora release stream (e.g. post-release
> updates, rawhide).  It's great that we have predictable schedules, and that
> we have predictable feature sets.  What sucks is that nobody knows how well
> those feature sets will work, and just as importantly, nobody knows what
> types of package updates (wrt ABI compat, rebases, etc) to expect in the
> Fedora update streams.  Things which work on release day may break later on
> due to some packages getting de-stabilizing updates, some not.  We ask
> maintainers to use their judgement, and when people complain, we explain
> that we don't control it, and it's up to the maintainer.  This clearly does
> not work as this is a frequent topic of contention on various mailing lists,
> and is not really beneficial to any class of user.
>
> Additionally, nobody knows whether rawhide on any given day will be usable,
> and it still has a negative stigma attached to it: some engineers who
> provide rawhide packages don't even use rawhide, which is causing rawhide
> and thus the upcoming releases to get less testing.  I do not think that we
> can really provide top quality software if we can't even use what we build
> at any given point.  We are more focused with getting features in then with
> getting them in and having them work.  We are happy to break things in
> rawhide and say "well, it's rawhide, i'll fix it in a few days".
>
> If we want to target Fedora for any class of user, we need to think and act
> for the user.  Right now, we're clearly not even acting for the people that
> do use our distribution.  I think we should fix that before we can even
> begin to define what our target user should be.  If we can't do these five
> things, then I think any discussion involving target users is rather
> pointless, and quite honestly, we are doomed to fail, IMHO. (Note that some
> of these items may be best specified by e.g. FESCo, but I think they reflect
> a significant enough change in the way we do things that the Board should
> push for and stand behind these).
>
>
>   1. Define a list of core/critical-path functionality that packagers are
> required to ensure they do not break.  Define a plan of action for what
> happens if such functionality becomes broken.  See example[1]. Bonus points:
> come up with an easy to follow "smoketest" for how to determine whether
> something on the list is broken.
>
>   2. Define update criteria for our release streams: what types of updates
> we expect, and what types of updates we do not want in each stream.  Define
> a plan of action for what happens if an update fails to comply.  See
> example[2].
>
>   3. Set up something similar to Mozilla's "Sheriffs"[3].  The Sheriff will
> be a rotating role and shall be responsible for coordination and enforcing
> of the previous two rules.  If an issue arises, the sheriff will attempt to
> contact the packager responsible, and attempt to get them to fix or revert
> the issue.  If this isn't possible within 15 minutes, the sheriff will find
> a provenpackager to do so.
>
>   4. Improve our test suites.  Provide $coolstuff incentive for people who
> contribute (the most?) valid test cases.
>
>   5. Start an initiative to automate as much of the above as possible.
>  Possibly as GSoC projects.  Particularly, I'd like to see a tinderbox which
> creates VMs from a buildroot+ks file, runs automated tests for the critical
> path, and outputs the PASS/FAIL results.  I'd also like to have a
> post-commit hook which reminds people to not break stuff and to be available
> to the sheriff on IRC.
>
>
> [1] Example: Core/critical-path is a system must boot up, get a display
> manager with XYZ video cards, be able to log in successfully, be able to get
> a working network via ethernet (and if available, via xyz wireless cards),
> have audio work on XYZ audio cards, and be able to successfully use
> yum/rpm/PackageKit.  In the event any package breaks this functionality, the
> package must be fixed immediately (within 15? minutes of noticing) or the
> changed is reverted, package untagged and rebuilt. If N violations occur,
> provenpackager status is revoked.
>
> [2] Example: For rawhide, do not break dependencies without announcing in
> advance about why you are doing so to fedora-devel-list, and not receiving
> objections.  For Fedora releases, updates must not break ABI or dependencies
> without getting an exception granted from FESCo.  In the event any package
> fails to comply, the change is immediately reverted, and mail sent to the
> packager owner.  If N violations occur, provenpackager status is revoked.
>
> [3] https://wiki.mozilla.org/Sheriff_Duty

Also see https://fedoraproject.org/wiki/Desktop/Whiteboards/UpdateExperience

Would be great to start working on these important problems.

Thanks,
Jon