how to improve the efficiency [Re: Round-up, 2004-11-28]

[Eric Rostetter]

Quoting Pekka Savola <pekkas at netcore.fi>:

> > See the security page.  Digital signatures are critical to the project.
> > No other way can one establish trust, and the project depends on a trust
> > model to succeed.
> 
> At least http://www.fedoralegacy.org/about/security.php does not
> describe this at all.

I completely disagree.  So can you tell me what it is you expect to see
there that you are not seeing there?

> Note that I'm not arguing that the RPMs should not be signed, but that
> the reviewers would not have to sign their comments if they didn't
> want to bother.  There is no gain in requiring that.

>From the above web page:

Like the previous fedora.us development, all Fedora Legacy developers are
required to use GPG in a proper manner in all communications of substance. This
allows messages and packages to be verifiable back to specific individuals. Most
importantly this enables building the corpus of historical evidence necessary
for anyone to read, verify GPG signatures, and decide based upon the
contributor's good advice if they are to be trusted or not. This is part of the
"web of trust" concept.

What we gain is someone not pretending to be others, reposting someone elses
posting under a different identity, someone being known/recognized as
a single individual even if they post under different names or email
addresses, and so on.  Plus we know the contents of the posting are not
corrupted, etc.  Plus we can correlate their postings on other sites to
ours, so we can build trust based on other projects they work on, etc.
 
> > Well, you refer to the link given, which tells you how and what to verify.
> 
> So, you'll only check the PGP signature?  That's simple, sure. If
> there is nothing more to do than to check that the PGP signature is OK
> and it installs nicely, why don't just say that?

No, it tells you both how to check the signature, and the md5/sha1 checksums.

If we say this in-line in the QA page, we need to include all the instructions
there which is several paragraphs and which are duplicated elsewhere.  So
instead, we point to a single page, which lists all the info.  Then if we
need to update things, we only need to update one page, not multiple pages.
 
> E.g.,
> 
> Verify the PGP signature of the downloaded package
> (see the details at http://www.fedoralegacy.org/about/security.php).

First, it would have to be:

Verify the PGP signature and md5 or sha1 checksum of the downloaded package
( see the details at blah blah blah).

I fail to see how that is reall much more clear than:

Verify the integrity of the downloaded package (see
http://www.fedoralegacy.org/about/security.php).

when that page then has a section called "Verifying RPMS packages" on it.

But we can make it more verbose, no problem.  I'll take care of that.
 
> As it is, the statement seems to imply there might be a lot more to
> verify than just PGP signature of the RPM.

There is.  You should also check the checksums as documented on the page.

> >> How do you report being successful after "4"?
> >
> > You see the next section of the page, IIRC.
> 
> Not sufficiently concise what must be done (an example might not
> hurt, e.g., a pointer to a bug and referring to the comment number to
> look at).

Please provide an example.  I'm not completely sure what you want.
Sounds to me like any such additional content would be another page which
this would link to (as an example page, for instance).  But I'm really not
sure what you mean.

> That's not good marketing. :)

Unfortunately I have no marketing experience.

> Unless the website can describe in a manner that can be understood in
> less than 5 minutes how you can help in the process, it doesn't help
> those people (most of them :) who don't have hours to devote to
> finding all the details -- at least at first.

One hopes that if they have problems or questions they will ask for help
on the mailing list or IRC, though I know many won't.
 
> The learning curve must be _very_ low.  That's only way to get
> participation.

I agree, and I agree it is slightly too high (but not as high as you think
it is).  But many of the people involved (myself, Jesse, etc) are very busy,
and our knowledge base is not in marketing or writing, so we need help.
We need people to step up and bit-wise refine the docs.  Tell us what is
bad, what needs work, what needs more explaination, etc.  And if possible,
give us actual text to add or replacement text to use instead.

> > Yes.  I wish the people with these scripts would put them up in the wiki
> > with instructions on how to use them, etc.  So far, no one has taken up
> > my challange to do that, AFAIK.
> 
> Yeah.. this is a BIG problem for doing meaningful upgrade testing.
> (Not so much at updates-testing phase, but in the previous stage).

If I get time, I'll try to contact people who posted scripts or url links
and get permission to put them in the wiki.  But it sure would be easier
if they authors would do this stuff themselves.
 
> >> There should also be more documentation at least on following:
> >>   1) documenting new vulnerabilities (I think Jesse has mostly been
> >> doing this) -- in bugzilla?
> >
> > What isn't covered about this?  If you tell me what is missing, I'll
> > add it.  If the problem is you don't know where to look, let me know
> > and we'll try to make some cross-links to make things easier to find,
> > or menu changes to make it easier.  But you need to tell us what the
> > problem is.
> 
> What kind of information do you need to put in the report?  How do you
> articulate the subject, what other special keywords you put there,
> etc.?

I can see where this info is missing.  I'll put it on the todo list.

> There seem to be two options:
>   1) someone really familiar with an optimal process jumps up to
> describe/organize it better for the FL QA beginners, or

First, some of this stuff is still to be decided.  So it doesn't even
exist yet, and hence can't be documented yet.

Second, no one with familiarity to the optimal process is going to jump
up any time soon, because all 4-5 people who fit this description are
too busy right now, etc.

BTW, I'm a FL QA beginner myself.

>   2) a FL QA beginner with a lot of free time comes up, figures out
> everything on his own, and then suggests fixes to the documentation.

First, he shouldn't have to figure it out on his own.  Help is available,
they just need to ask for it.

But yes, the way the docs got written is myself and two other beginners
who had no clue started trying to figure it out and document it.  If another
2 or 3 would join in, we could make real progress.  The current docs 
present the problems and solutions the 3 of us original people had joining
the project.  It met our needs.  If it doesn't meet other people's needs,
the best way to fix that is that they speak up.

But they can't speak up in vague terms (the docs suck, the docs are not
complete, the docs are too vague).  They need to speak up in specific
terms (the docs on doing X are too vague, or I can't find page X from
page Y, or paragraph 5 on page Z is unclear).

> This would take a LOT of time (I'd say in the order of week), so I
> fear this isn't going to happen.

Well, it may, or may not.  But if you just post a specific question or
complaint, then we can fix it, or explain why it shouldn't be fixed,
and one more thing is improved.  One step is better than no steps.

> I'd really hope that someone from category 1) could try to re-think
> the documentation from a FL tester's perspective

They have.  But from a more experienced FL tester's perspective.  Once
you pass from beginner to intermediate or advanced, it is *very* hard
to look at it again from a beginner's point of view.  For that, we need
additional beginners...

> -- and when it would
> be much better, later the the huge influx of FL testers (yeah, right
> ;-) could also step up and suggest minor enhancements.

It really all starts with minor enhancements.  Major overhauls rarely
work. Minor, incremental fixes and enhancements by multiple people
are almost always the best way to improve documentation and web sites.

Plus, one person's discussion often leads others to join in, and results
can really come fast from there.  But we need that first person to
"jump start" the process.

> --
> Pekka Savola                 "You each name yourselves king, yet the
> Netcore Oy                    kingdom bleeds."
> Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

-- 
Eric Rostetter