Updating RPMs using binary deltas (demo)

[Alexandre Oliva]

On Jan 28, 2004, Leonard den Ottolander <leonard at den.ottolander.nl> wrote:

> Just always delta against the original rpm that
> came with the release, since everybody should have that one lying around

Lying around in the install CD?  Or in .iso format on some NFS server
used for the install?  I guess these are the most common cases, and
I somehow can't see them as convenient.

The most common case is that the person will the previous release of
an update installed on their system.  If you only generate deltas
from the base release, you don't help such people.

If you generate deltas from base to first update, from first update to
second, etc, it works but you may not save as much on downloads.  If
you generate deltas from each earlier release to the latest, you save
on downloads, but potentially waste disk space.

Using an rsync-based download, OTOH, you don't need to generate deltas
at all: you pre-generate the hashes on the server, the client
downloads that, figures out which chunks it needs based on what it has
on its end of the network, and requests only those from the server.

>> A simplistic rsync-based solution OTOH just requires hashes for
>> every package to be pre-computed on servers and made available for
>> download.

> Rsync puts a load on the server, which is probably not what you want.

See the `pre-computed hashes' bit.

> These delta's (have you tried these scriptlets yet?) can just be
> downloaded.

The problem with the deltas is that they're static, making a guess on
what the client has on its end.  rsync hashes, OTOH, make no such
assumptions: they tell, with a very small footprint, what the client
is about to download, such that the client can tell which bits it
doesn't need.  The fact that rpm uses minigzip makes it as suitable
for rsyncing as for xdelta.

> These delta bundles have to be created only once and can be distributed
> beside the normal rpms.

Just like the rsync hashes.  But the rsync hashes are based only on
the one (updated) binary rpm you're willing to distribute.  No
assumption on what the base version was.  So it can be used for
upgrades as much as it can be used for updates, without the overhead
of generating xdeltas from every other earlier distro release,
regardless of released updates.

> As I mentioned above, the "local tree" should contain the initial rpm
> for 99% of the users. No need to put any load on the servers.

Not necessarily true for the half dozen kernel updates we've had in
FC1 since its release.  kernel-sources is pretty big, and up2date -u
replaces the installed files and then throws the .rpm file away.

The rsync protocol-based solution I'm proposing would reconstruct the
rpm file from the installed files, and use that to save on the
download.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Happy GNU Year!                     oliva@{lsd.ic.unicamp.br, gnu.org}
Red Hat GCC Developer                 aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist                Professional serial bug killer