[fedora-virt] Re: Spec weirdness

Mon Aug 10 18:45:04 UTC 2009

On Mon, Aug 10, 2009 at 10:40:16AM -0700, Jesse Keating wrote:
> I'm looking at the spec file for libguestfs, and all I can say is WTF.
> There is a lot of crazyness going on in this spec, chroots within
> chroots, making a repo of yum cache packages and using it again, calling
> qemu, and none of it is really documented in the spec as for what and
> why things are done this way.

Hey Jesse, I'll explain below what the general idea is, then provide
at the end some commentary inline on specific points within the
specfile.  It certainly is a complex specfile.

In case you aren't familiar with it, libguestfs (http://libguestfs.org)
is a tool for modifying VM disk images.  For example:

  $ guestfish -a Fedora10.img

  Welcome to guestfish, the libguestfs filesystem interactive shell for
  editing virtual machine filesystems.

  Type: 'help' for help with commands
        'quit' to quit the shell

  ><fs> run
  ><fs> list-devices 
  /dev/vda
  ><fs> list-partitions 
  /dev/vda1
  /dev/vda2
  /dev/vda3
  ><fs> lvs
  /dev/VolGroup00/LogVol00
  /dev/VolGroup00/LogVol01
  ><fs> file /dev/VolGroup00/LogVol00 
  Linux rev 1.0 ext3 filesystem data (large files)
  ><fs> mount /dev/VolGroup00/LogVol00 /
  ><fs> cat /etc/redhat-release 
  Fedora release 10 (Cambridge)

  ><fs> vi /etc/passwd
  ><fs> ^D

What we do to make this happen is run a small Fedora distro (called
the "appliance") under qemu which is attached to the disk image in
question.  The appliance runs a Linux kernel, LVM tools etc etc and
thus has full access to the disk.  You can also run qemu as non-root,
which means you don't need to be root to edit disk images.

The complexity in the specfile is mainly around building this
appliance.

We use a tool called febootstrap which is basically a shell script
that runs 'yum' as non-root, using Debian's fakeroot and fakechroot
programs.

febootstrap can take any yum repository and turn it into an appliance,
without needing root permission:

  $ febootstrap fedora-11 ./f11
  $ febootstrap-to-initramfs ./f11 > initrd.img

Of course during the Koji build we don't have a yum repository, and we
don't have (or want) network access.  So we make one using createrepo
+ the BuildRequired rpms from the mock cache.

There's a further complexity here -- a normal appliance would actually
contain the kernel + glibc + libraries + programs.  However that is
undesirable from a Fedora point of view, since it's very similar to
static linking.  If we really shipped that as an appliance then there
would be obvious security / patching headaches if (for example) glibc
had a security bug.  Therefore we don't actually bundle any binaries.
We remove those from the final appliance, and get them from the host
system at runtime.  The technique is called building a "supermin
appliance" and it's described here: http://libguestfs.org/README.txt
under "supermin appliance" It is important to reiterate that the
appliance we ship in the RPM *does not* contain any libraries or
binaries.  We just remember where those files were, and pull them out
of the host's filesystem at runtime.

>From the point of view of improving Koji (or mock) it would be nice to
have a more official method to access the build RPMs, although the
technique we are using works fine.

On to the specfile itself:

  # Enable to build w/o network.
  %global buildnonet 1

  Summary:     Access and modify virtual machine disk images
  Name:        libguestfs
[...]  
  # Basic build requirements:
  BuildRequires: /usr/bin/pod2man
  BuildRequires: /usr/bin/pod2text
  BuildRequires: febootstrap >= 2.3
  BuildRequires: augeas-devel >= 0.5.0
  BuildRequires: readline-devel
  BuildRequires: squashfs-tools
  BuildRequires: qemu-kvm >= 0.10-7
  BuildRequires: createrepo
  BuildRequires: glibc-static

  # This is only needed for RHEL 5 because readline-devel doesn't
  # properly depend on it, but doesn't do any harm on other platforms:
  BuildRequires: ncurses-devel

These are the basic build requires for making the actual package.

  # Build requirements for the appliance (see 'make.sh.in' in the source):
  BuildRequires: kernel, bash, coreutils, lvm2, ntfs-3g, util-linux-ng
  BuildRequires: MAKEDEV, net-tools, augeas-libs, file
  BuildRequires: module-init-tools, procps, strace, iputils
  BuildRequires: dosfstools, zerofree, lsof, scrub
  %ifarch %{ix86} x86_64
  BuildRequires: grub, ntfsprogs
  %endif

These are the BRs reflecting what is inside the appliance ...

  # Must match the above set of BuildRequires exactly!
  Requires:      kernel, bash, coreutils, lvm2, ntfs-3g, util-linux-ng
  Requires:      MAKEDEV, net-tools, augeas-libs, file
  Requires:      module-init-tools, procps, strace, iputils
  Requires:      dosfstools, zerofree, lsof, scrub
  %ifarch %{ix86} x86_64
  Requires:      grub, ntfsprogs
  %endif

... and at runtime we need the same set of Requires, because of the
supermin appliance described above.  Those files that we evicted from
the appliance at build time must exist in the host system at runtime.

[...]
  %build
  %if %{buildnonet}
  mkdir repo
  find /var/cache/yum -type f -name '*.rpm' -print0 | xargs -0 cp -t repo
  createrepo repo
  %define extra --with-mirror=file://$(pwd)/repo --with-repo=fedora-12 --with-updates=none
  %else
  %define extra %nil
  %endif

This is the part where we create the local repository from the mock
cache.  The %{buildnonet} switch allows people to build from the
Fedora repos instead if they do have network access.

  # --with-net-if=ne2k_pci is a workaround for RHBZ#516022.
  ./configure \
    --prefix=%{_prefix} --libdir=%{_libdir} \
    --mandir=%{_mandir} \
    --with-qemu="qemu-kvm qemu-system-%{_build_arch} qemu" \
    --enable-debug-command \
    --enable-supermin \
    --with-net-if=ne2k_pci \
    %{extra}

  # This ensures that /usr/sbin/chroot is on the path.  Not needed
  # except for RHEL 5, it shouldn't do any harm on other platforms.
  export PATH=/usr/sbin:$PATH

  # 'INSTALLDIRS' ensures that perl libs are installed in the vendor dir
  # not the site dir.
  make INSTALLDIRS=vendor %{?_smp_mflags}

This bit is building the library + appliance + 6 language bindings.

  %check
  # Enable debugging - very useful if a test does fail, although
  # it produces masses of output in the build.log.
  export LIBGUESTFS_DEBUG=1

  # Uncomment one of these, depending on whether you want to
  # do a very long and thorough test ('make check') or just
  # a quick test to see if things generally work.

  # Tracking test issues:
  # BZ       archs        branch reason
  # 494075   ppc, ppc64          openbios bug causes "invalid/unsupported opcode"
  # 504273   ppc, ppc64          "no opcode defined"
  # 505109   ppc, ppc64          "Boot failure! No secondary bootloader specified"
  # 502058   i386, x86-64 F-11   need to boot with noapic (WORKAROUND ENABLED)
  # 502074   i386         F-11   commands segfault randomly
  # 503236   i386         F-12   cryptomgr_test at doublefault_fn
  # 507066   all          F-12   sequence of chroot calls (FIXED)
  # 513249   all          F-12   guestfwd broken in qemu (FIXED)
  # 516022   all          F-12   virtio-net gives "Network is unreachable" errors
  #                                 (WORKAROUND ENABLED)
  # 516096   ?            F-11   race condition in swapoff/blockdev --rereadpt
  # 516543   ?            F-12   qemu-kvm segfaults when run inside a VM

  #%ifarch x86_64
  #make check
  #%endif

Normally we run the extensive test suite.  However because of a bug in
qemu (#516543) we are unable to run that at the moment.

And the rest is just concerned with the language bindings.

Rich.

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top