preupgrade today: getting error: cannot download the kickstart file

[John Reiser]

Matt Domsch wrote:
> On Sat, Mar 28, 2009 at 09:18:42PM -0700, John Reiser wrote:

>> For instance, by default yumdownloader generates no record of which mirror
>> you get upon HTTP 302 [or 303] redirection from the master mirror.
>> In practice you see the ultimate URL only if the connection [attempt]
>> actually times out.
> 
> Patches or enhancement bugs filed suggesting ways in which the tools
> you're interested would be more verbose are welcome.
> 
>> This enables the mirror administrator(s) to hide from users in most
>> cases of poor service (missing file, stale file, slower than
>> necessary, ...)
> 
> This, my friend, is just a pot shot.  We're _lucky_ to have so many
> mirrors, and in most cases, mirrors with very responsive
> administrators; not to mention the Infrastructure team that keeps it
> all humming.  If you have genuine problems with particular mirrors,
> feel free to let the folks in #fedora-admin know, or file a ticket in
> the Fedora Infrastructure trac
> (https://fedorahosted.org/fedora-infrastructure) with the details.

As a mostly end-user of the mirror system, I consider the remark
to be on-target and as specific as possible with the current mirror
system and usage.  It is not a pot shot.

Rescue mode and install from physical DVD are important to me.  I try
to help make them better by testing during development.  During the
alpha period for Fedora 11, I have composed install DVD via pungi
from rawhide on about 20 days.  Usually I try both i386 and x86_64.
About 40% of the time things work very well.  The downloading required
by the compose goes quickly, and testing the resulting DVD verifies
a fix, sheds new light on a continuing problem, or reveals a new bug.
About 40% of the time things go OK: average download speed, perhaps
a snag that is easily fixable, perhaps progress on rescue mode, etc.
About 20% of the time the results are poor.  Downloading takes a long
time, fails, or fetches stale data; or the compose or booting or rescue
fails, sometimes in mysterious ways.  The exact cause is murky, and
frustrating to determine.  The mirror system ought to be near the bottom
of the list of suspects, but it isn't.  Somehow it feels like the mirror
system is providing about 90% over-all reliability when I'm expecting 97-99%.

I start the pungi compose a few hours after receiving the rawhide report.
As long as there are fewer than 100 packages or so, then I consider this
to be time enough for changes to propagate through the mirror system.
I precede the compose with "yum update" to check for changes to pungi,
anaconda, yum, mkinitrd, and rpm; and if found then I update them before
the compose.  Other packages wait for update until after the compose,
which is done as yum localupdate from /var/cache/pungi/rawhide/packages.
[Thus, a very low-tech mirror for propagating daily changes to two places.
My connection is cable modem with 12Mbit/s advertised but often <= 1MByte/s.]

What do I see?  Many times success, but too often murk.
Different packages changed from rawhide report to "yum update" to pungi
download, and not just the big spikes for translation string updates,
mass keyed signings, deliberate release holds, etc.  Sporadic delays
of 10 seconds or more for establishing connections, as viewed in System
Monitor.  Throughput varying from 80KB/s to 3MB/s for observable
durations on non-small files.  Most importantly: little indication of why,
and little hope of finding out.

Pungi does report "Connection timed out" so that is obvious.  The log file
sometimes contains one of:
  Requested Range Not Satisfiable
  Package does not match intended download
  HTTP Error 404: Not Found
with an associated URL.  That's good, but the same message may appear
consecutively for more than one mirror.  Where does the blame lie: with
each listed mirror, or with the master, or with pungi (or its delegate)?
Which mirror gets credit for succeeding with this package,
thus *NOT* appearing in a complaint at the moment?  When the package
that is downloaded is not the version listed in rawhide report: why?

The mirror system provides me with much good use, but enough not-so-good
use that I grouse about it.

--