Automated Mirror Selection [Re: Worst experience with Up2Date ever.]

[Jeff Spaleta]

On Wed, 22 Sep 2004 22:28:35 +1000, Rodd Clarkson
<rodd at redfishbluefish.com.au> wrote:
> The icon in the notification area is currently displaying as red and
> shows that 55 packages need updating, but if I click on the icon and
> then click Launch up2date (which shows 55 packages needing updates) and
> then click through, up2date says my system is up to date (it isn't)
> 
> It's trying to use yum channel fedora-core-rawhide from
> http://download.fedora.redhat.com/pub/fedora/linux/core/development/i386/
> for the updates.

I bet you also have an active yum-mirror fedora-core-rawhide  line as well.
Assuming you do have a yum-mirror line active, what are are most
likely seeing is what happens mirrors are out of sync with the master
download server.  The panel applet( rhn-applet-gui) sees there are
updates available from the main server i believe. But when up2date is
run, up2date does some sort of selection among the mirrors in the
mirrorlist
( http://fedora.redhat.com/download/up2date-mirrors/fedora-core-rawhide
)  chooses a mirror to try to download packages from and then attempts
to download the packages.

So what i think you are seeing is a side-affect of the fact that
different mirrors sync with the main download server at different
rates and times.  If this is a case, this is just one example of how
hard it is to do reasonable mirror selection without requiring
information to flow from a mirror back to a centralized location about
the sync status of each mirror, or by requiring the end-user to make a
selection on the client-side to choose mirrors based on past
experience.

We could have many, long and fruitless discussion (and I have) about
how to do mirror-selection better given the current constraints on how
the mirrors are setup.  So before someone decides to run off on a
tagent about how to do it better than up2date does it right now. I'm
going to set down the constraints that define the current mirror
structure. If you start a discussion that does not live with in the
boundaries of these constraints you're fantasizing about an
unimplementable utopia that does not match the current political and
technical realities which you can't change. And i will publicly point
and laugh at you and call you names if you attempt to do so.

1) the mirror networks are NOT peer-to-peer nodes. Anything that
requires a mirror to run a specialized daemon is not going to be
acceptable. One of http/ftp/rsync are the only services mirrors need
to choose to run.

2) No mirror-side scripts to communicate back to a centralized
location to report that the mirror has started or is done syncing to
the master server. No information is "pushed" from a mirror back to
the master mirror or to another centralized location. Any statistics
gathering you might think of doing to try to determine if mirrors are
in sync with the master mirror needs to be done by a seperate server
and not services implemented at each mirror.

3) No scheme can rely on red hat specifically implementing a new
centralized service or providing the infrastructure for a new
mirror-selection service. If you do come up with a clever way to
provide accurate list of the mirrors that are in sync with the master
mirror, build it on a community controlled site and get people testing
it with up2date. There is no reason up2date's yum-mirror  has to take
a redhat.com address as an argument.

-jef