[Fedora-xen] sparse-merge tutorial

[Aron Griffis]

Here is the process I use for porting xenlinux to the kernel version
used by fedora rawhide.  I suspect this is very similar to what Juan
is doing, but I haven't talked with him yet about it.

Aron Griffis wrote:  [Sat May 20 2006, 03:34:35PM EDT]
> 1. Upstream xen is presently based on 2.6.16.13.  Fedora kernel is (or
>    was yesterday) based on 2.6.17-rc4-git5.  To port xen forward, the
>    most maintainable method seems to be to do the port in the context
>    of a xen-ia64-unstable mercurial clone (xen-ia64-unstable-2.6.17
>    above).  Using this method makes it relatively easy to:
> 
>     (a) port forward to a new kernel at any time using the
>         sparse-merge script
> 
>     (b) pull new changes from upstream xen and avoid most manual
>         merging
> 
>     (c) extract a patch at any time that represents the forward-port
>         of xen to a new kernel
> 
>     (d) generate a patch at any time that adds xen support to the
>         fedora kernel (linux-2.6-xen.patch generated with "make
>         mkpatches")

First, determine what version the Fedora kernel is based on.  Get the
src.rpm and look inside it to see what version it is based on:

$ wget http://download.fedora.redhat.com/pub/fedora/linux/core/development/SRPMS/kernel-2.6.16-1.2208_FC6.src.rpm
$ mkdir tmp
$ cd tmp
$ rpm2cpio ../kernel-2.6.16-1.2208_FC6.src.rpm | cpio -imd
$ ls *.bz2
linux-2.6.16.tar.bz2
patch-2.6.17-rc4-git9.bz2
patch-2.6.17-rc4.bz2
xen-20060503.tar.bz2

So we see that 2208 is based on 2.6.17-rc4-git9.  We will need
a tarball for xen to unpack later, so make it now:

$ tar xjf linux-2.6.16.tar.bz2
$ mv linux-2.6.16 linux-2.6.17-rc4-git9
$ cd linux-2.6.17-rc4-git9
$ bzip2 -dc ../patch-2.6.17-rc4-git9.bz2 | patch -p1
$ cd ..
$ tar cjf ../linux-2.6.17-rc4-git9.tar.bz2 linux-2.6.17-rc4-git9

Save off the patch for later use, and remove the temporary directory:

$ mv patch-2.6.17-rc4-git9.bz2 ..
$ cd ..
$ rm -r tmp

Next clone a mercurial kernel tree.  This will be for the sparse-merge
script to reference:

$ hg clone http://kernel.org/hg/linux-2.6

And clone the xen repo you want to use (xen-ia64-unstable or
xen-unstable):

$ hg clone http://xenbits.xensource.com/ext/xen-ia64-unstable.hg

Next use the sparse-merge script to port the tree forward.  Don't
forget the "v" in NEWTAG.  OLDTAG is fetched automatically from
buildconfigs/mk.linux-2.6-xen.  ARCH defaults to ia64, so override it
to "." to match all files in the sparse tree.

$ cd xen-ia64-unstable.hg
$ LINUXDIR=$HOME/linux-2.6 NEWTAG=v2.6.17-rc4-git9 ARCH=. \
    xen/arch/ia64/tools/sparse-merge

At this point sparse-merge will do its magic.  For the most part, the
port will go forward smoothly, but it will also announce rejects as it
goes.  When it finishes, find the rejects and fix each one
appropriately:

$ find linux-2.6-xen-sparse -name \*.rej

When you're done, you can remove the rejects and diffs from the sparse
tree.  You might save them first in case you'd like to reference them
later:

$ find linux-2.6-xen-sparse -name \*.rej -o -name \*.diff \
    | xargs -r tar cjf ../diffs-rejs-2.6.17-rc4-git9.tar.bz2
$ find linux-2.6-xen-sparse -name \*.rej -o -name \*.diff \
    | xargs -r rm -f

Next update the buildconfig to reflect the new version that xen is
based on:

$ sed -i '/^LINUX_VER/s/=.*/= 2.6.17-rc4-git9/' \
    buildconfigs/mk.linux-2.6-xen

Next do a test build:

$ make world

If it fails, you'll have to investigate and then fix things in the
sparse tree, then do:

$ make kdelete world

You can do this until you get a clean build, then it's time to commit
your changes and make the patch:

$ hg add linux-2.6-xen-sparse
$ hg remove --after linux-2.6-xen-sparse
$ hg st
$ hg ci -m "sparse-merge to 2.6.17-rc4-git9"

$ make mkpatches

At this point you have a patch which should apply to the Fedora
kernel.  Here is what I do personally:

$ cd $HOME
$ hg clone http://free.linux.hp.com/~agriffis/fedora-kernel-ia64
$ cp xen-ia64-unstable.hg/linux-2.6-xen.patch \
    fedora-kernel-ia64/SOURCES/linux-2.6-xen-ia64-10138.patch
$ rpmbuild -ba fedora-kernel-ia64/SPECS/kernel-2.6.spec

This assumes the patch was made based on (upstream) xen-ia64-unstable
cset 10138, and doesn't count any spec changes you might need to make.

I hope this helps.  At this point it's easy to either update the patch
within the existing tree for a new kernel version using sparse-merge,
or update to a newer xen-ia64-unstable cset with hg pull, hg merge.

Hope that helps,
Aron