Back Again

[Todd Zullinger]

Thank you for the detailed reply Sam. :)

Excuse me for asking more questions (potentially dumb questions at
that).

Sam Varshavchik wrote:
> I don't know if you've ever upgraded Fedora from one release to the
> next.  The upgrade process is as slow as molasses, even though all
> the metadata is right there.

No, I avoid upgrades.  I always do fresh installs as a matter of
habit.  Point taken though.  I have read a lot of complaints of slow
upgrades at the dependency resolution stage.

> A few years ago the base distro was much smaller than it it now. The
> size of a typical Linux distro has really balooned. Some of the
> algorithms in rpm scale horribly. It wasn't such a big deal when a
> typical linux distro was only a few hundred packages, but now it's a
> few thousand packages, with dependencies that are much more
> complicated, and rpm is now really blowing apart at the seams.

I haven't looked at the code, but is it rpm or yum that's really
bogging down?  Or aren't you making much of a distinction when you say
rpm?

> Furthermore, rpm, as is, does not implement remote repositories.

Does it need to?  Does dpkg do this?

> With a large repository, like Fedora, even a compressed XML file is
> going to end up being rather huge. Then, you have to uncompress it
> and parse it.  And, XML parsing is also not exactly a light task.

But somehow or another you need to deal with a sizable chunk of data
to make reasonable decisions regarding dependencies.  The tough part
about rpm development is trying to be backward compatible and still
make forward progress.  I don't envy the guys hacking on rpm.

> Remote package repositories could've been implemented much better.
> When I had some free time some time ago, I quickly hacked up a
> package manager for some of my internally-developed software. I
> found that I could do similar kind of package metadata
> synchronization much more efficiently than yum/rpm.

Isn't the harder part doing this in a way that doesn't completely
break backward compatibility though?  And then you have to spend a
bunch of years adding new code to deal with the odd sorts of deps that
packagers come up with in the wild (versioned obsoletes on a multilib
system sounds fun :).

Someone posted to the fedora-devel list a month or so ago saying
they'd created a super fast depsolver using php and mysql.  Once all
of the various cases they'd missed were explained, things didn't go
much further.  (And no, I'm not at all suggesting that applies to your
work -- it's obvious that you know more than that and that you
actually created a working system. :)

> Using some tricks that involve HTTP 1.1's partial chunking features,
> you can implement synchronization of local/remote package repository
> metadata using a constant number of round trips, no matter how many
> packages you need to download the metatadata for. Rather than having
> to download a verbose XML file that enumerates the packages in the
> repository in a rather verbose format, all you need to start are a
> flat file with their names, which is much smaller. Then, once you
> have a tiny list containing, basically, the package names and not
> much more, you can quickly identify which packages' metadata you do
> not have cached locally, and take exactly one more roundtrip to the
> server, using HTTP 1.1 partial chunking, to download the metadata
> for all of them, at once. On the server you basically keep all the
> metadata in one flat file, just as yum does now, but if you already
> know which packages you want, and you know which parts of the
> metadata file you want to download, you can use HTTP 1.1 partial
> chunk request feature to download just the bits of the metadata file
> that you want.

Perhaps you should bribe someone to implement this in yum as a proof
of concept?

> But then, after all is said and done, no amount of tweaks to rpm can
> compensate for stupid and broken packaging. Right now, due to
> indirect dependencies, grub requires *GTK* runtime libraries to be
> installed. On my headless machine, I now have to plop down a
> crapload of x.org and GTK RPMs, because grub requires them, due to
> its intermediate dependencies.

Yeah.  This was caused by policy more than by incompetence.  The folks
at Red Hat's legal department asked that all of the trademarked logos
be kept in one package, for easier tracking and removal by downstream
users of Fedora's packages (or something like that).

> And of course, you need grub to boot Fedora, so you can't do without
> it.

Not that you should have to, but you could install lilo instead. :)

> This is stupid: grub requires the fedora-logos rpm. The fedora-logos
> rpm requires the redhat-artwork rpm.  redhat-artwork requires
> gnome-themes.  gnome-themes requires gtk2-engines. gtk2-engines
> requires gtk2, and a crapload of gtk and xorg rpms.
>
> This is insane. There's a bug filed about this, but nobody seems to
> care.

Someone cares.  It's actually been fixed in rawhide AFAIK.  See the
CVS commit:

http://cvs.fedora.redhat.com/viewcvs/rpms/fedora-logos/devel/fedora-logos.spec?rev=1.68&view=markup

and this post on fedora-devel:

https://www.redhat.com/archives/fedora-devel-list/2007-June/msg01904.html

Thanks again for the informative reading!

-- 
Todd        OpenPGP -> KeyID: 0xBEAF0CE3 | URL: www.pobox.com/~tmz/pgp
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Never attribute to malice that which can be adequately explained by
stupidity.
    -- Hanlon's Razor

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 542 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/fedora-list/attachments/20070731/efd48e49/attachment-0001.sig>