[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Proposals for the Updates Testing Procedure



I am glad to see many test updates being announced on fedora-test-list, but may I please suggest some improvements to this mechanism. We also need to discuss the eventual creation of policy for the update approval process.

Along with each announced test update, I believe it would be crucial to include a link to a corresponding Bugzilla report. While longer discussions pertaining to packages can remain on fedora-test-list, all pertinent information should be posted in one place within that test update's Bugzilla report. Why?

* Otherwise it is likely that other Bugzilla reports and important information related to an update could easily be lost in the mailing list noise.
* All pending updates can easily be found by a single Bugzilla database query.
* Each report would become a one-stop-shop for information regarding that update. More responsible sysadmins with proper testing procedures can read that report to help in their decision to update.



Discussion of the Update Approval Process
=========================================
We also have not yet discussed the fedora.redhat.com update approval process. I suggest the following to begin the necessary discussion. Some of the below are similar in concept to the procedures currently being used successfully at fedora.us as described in this document:
http://www.fedora.us/wiki/PackageSubmissionQAPolicy


* Backport Patches v. Upgrade Version
Most libraries, core system and server packages should always have backport patches. End-user applications where other packages do not depend on them can possibly be upgraded in version after sufficient testing in updates-testing. There are always possible exceptions to the rule, although the determination of backport verses upgrade version will need to be decided on a case-by-case basis based upon how intrusive the change would be. (Yeah this description sucks. Reply with a better and expanded proposal if you care about this.)


* Time-limit to publish where no negative comments are posted within the Bugzilla report. Senior developers reserve the right to hold an update if a good technical reason can be stated. (Insert more details here.)

* GPG clearsigned messages of approval
For the test update approval process, a sizable number of people who matter (this needs further discussion) should post GPG clearsigned messages of approval to the Bugzilla report for the pending update currently in testing. Such messages should be posted if they technically agree with the patch/upgrade, and they have fully tested the functionality of the updated binary RPM and are satisfied that it is better than the previous package. Otherwise dissenting messages about functionality brokenness or suggestions for further package improvement are posted.


http://www.fedora.us/wiki/QAChecklist
There should be a checklist similar to this one used at fedora.us that contributors must go through and say "Passes all checklist items." within their report. This checklist idea has successfully prevented many common problems from being published in fedora.us. Depending on the criticalness of the update, the release managers decide when it is the appropriate time to publish based upon proper & signed contributor feedback.


Before judging the above to be too an "onerous waste of time" as is the reaction that many people have, do know that this type of process has worked well at fedora.us during this year. This fedora.us report below is a great example of this type of process in action:

https://bugzilla.fedora.us/show_bug.cgi?id=669
Well... something like this report, except a Fedora Core update would have fewer package updates during the approval process, and far more users saying "gcc-3.3.2-2 works for me". (It is important that clearsigned messages contain unique identifiers like the full package name-version-release because messages like "works for me" alone can be abused through a replay attack.)


Why GPG? Wouldn't that be slow and a waste of time?
====================================================
No. The GPG clearsigned messages over time that collect within many reports and mailing list posts help to build a mass of "historical evidence" about the reliability and trustability of the advice given by a contributor. Over time this actually improves the efficiency of communication in a massively distributed project like one we are trying to create because of the following reasons.


1. Developers and users have a way to quickly search a history of a contributor's posts and contributions. That person may then quickly form an opinion about the skill level or reliability of the advice giver.
2. GPG signatures make it a lot more likely that the message came from the signer... the actual user holding the key. If that key is ever stolen, it would likely be discovered soon enough by other developers or the key owner if fraudulent messages are being posted.
3. Documented HOWTO procedure documents in GPG usage and proper use of messages give new contributors (developers, testers, etc.) a structured way to begin to build credibility and ease their way into the project.
4. The corpus of good GPG signed technical advice builds relationships of trust between developers and users. This could potentially form the basis of a developer certification and nomination of community leaders in the future.



In order for GPG to become a standard for the project, we must have improved documentation with many examples of usage so newbies can quickly understand how it works. Perhaps better tools that interact with the X clipboard would help to improve the speed and ease of use of GPG clearsigning of messages too. Can anybody recommend existing tools that may help in this?


Warren Togami
warren togami com




[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]