yum differential updates

[Gregory Maxwell]

On 4/10/06, Jesse Keating <jkeating at j2solutions.net> wrote:
> The problem is that even if it is a small change, all the source gets
> recompiled, so the resultant binaries could be different, and different
> enough to cause large amounts of little changes.  This is because we
> don't just spin the same binaries with say a new doc file or something
> like that.  Little changes just from a recompile add up quickly.

I thought it might be fun to inject a bit of data into this discussion.

First, a sort of worst case for a binary... libgcj from FC4 binary
delta compressed to the FC5 binary. (prelinking removed):

[gmaxwell at gmaxlpt ~]$ xdelta30d/xdelta3 encode -S djw -s
./libgcj.so.6.0.0 ./libgcj.so.7.0.0 libgcj.so.7.0.0.xd3
[gmaxwell at gmaxlpt ~]$ ls -l libgcj.so.7.0.0.*
-rw-rw-r--  1 gmaxwell gmaxwell 6226267 Apr 11 05:46 libgcj.so.7.0.0.gz
-rw-rw-r--  1 gmaxwell gmaxwell 4342762 Apr 11 05:47 libgcj.so.7.0.0.lzma
-rw-rw-r--  1 gmaxwell gmaxwell 6941616 Apr 11 05:42 libgcj.so.7.0.0.xd3

Although xdelta3 uses a BWT based compressor it doesn't have a large a
blocksize as bzip -9 , otherwise we'd see the xd3 file be almost
exactly the same size as the bz2.

How about an easy one..  Yum pulled a ghostscript update which changed
libgs without revving versions. Again, with prelink undone:

[gmaxwell at gmaxlpt ~]$ xdelta30d/xdelta3 encode -S djw -s
libgs.so.8.15.old ./libgs.so.8.15 libgs.so.8.15.xd3
[gmaxwell at gmaxlpt ~]$ ls -l libgs.so.8.15*
-rwxr-xr-x  1 gmaxwell gmaxwell 4830780 Apr 11 05:29 libgs.so.8.15
-rwxr-xr-x  1 gmaxwell gmaxwell 1615817 Apr 11 05:29 libgs.so.8.15.bz2
-rw-rw-r--  1 gmaxwell gmaxwell 1688814 Apr 11 05:53 libgs.so.8.15.gz
-rw-rw-r--  1 gmaxwell gmaxwell 1334810 Apr 11 05:35 libgs.so.8.15.lzma
-rwxr-xr-x  1 gmaxwell gmaxwell 4830780 Apr 11 05:30 libgs.so.8.15.old
-rw-rw-r--  1 gmaxwell gmaxwell  922597 Apr 11 05:52 libgs.so.8.15.xd3

So yes.. there is a gain for executables in what is likely the best
case (excluding the case where the file wasn't actually changed), but
it's not huge.

Not that delta compression is useless, consider a document which has
only had a date change:
[gmaxwell at gmaxlpt ~]$ ls -l slib.html.fc5*
-rw-r--r--  1 gmaxwell gmaxwell 1364094 Apr 11 06:04 slib.html.fc5
-rw-r--r--  1 gmaxwell gmaxwell  161391 Apr 11 06:04 slib.html.fc5.bz2
-rw-rw-r--  1 gmaxwell gmaxwell  160475 Apr 11 06:05 slib.html.fc5.lzma
-rw-rw-r--  1 gmaxwell gmaxwell     263 Apr 11 06:06 slib.html.fc5.xd3

or other simple incremental changes:
[gmaxwell at gmaxlpt ~]$ xdelta30d/xdelta3 encode -S djw -s
/usr/share/doc/gnome-vfs2-2.10.0/ChangeLog
./gnome-vfs2-2.14.0-ChangeLog gnome-vfs2-2.14.0-ChangeLog.xd3
[gmaxwell at gmaxlpt ~]$ ls -l gnome-vfs2-2.14.0-ChangeLog*
-rw-r--r--  1 gmaxwell gmaxwell 969090 Apr 11 06:10 gnome-vfs2-2.14.0-ChangeLog
-rw-r--r--  1 gmaxwell gmaxwell 176919 Apr 11 06:10
gnome-vfs2-2.14.0-ChangeLog.bz2
-rw-rw-r--  1 gmaxwell gmaxwell  26015 Apr 11 06:11
gnome-vfs2-2.14.0-ChangeLog.xd3