[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[fedora-virt] Re: Spec weirdness



On Mon, 2009-08-10 at 19:45 +0100, Richard W.M. Jones wrote:
> On Mon, Aug 10, 2009 at 10:40:16AM -0700, Jesse Keating wrote:
> > I'm looking at the spec file for libguestfs, and all I can say is WTF.
> > There is a lot of crazyness going on in this spec, chroots within
> > chroots, making a repo of yum cache packages and using it again, calling
> > qemu, and none of it is really documented in the spec as for what and
> > why things are done this way.
> 
> Hey Jesse, I'll explain below what the general idea is, then provide
> at the end some commentary inline on specific points within the
> specfile.  It certainly is a complex specfile.
> 
> In case you aren't familiar with it, libguestfs (http://libguestfs.org)
> is a tool for modifying VM disk images.  For example:
> 
>   $ guestfish -a Fedora10.img
>   
>   Welcome to guestfish, the libguestfs filesystem interactive shell for
>   editing virtual machine filesystems.
>   
>   Type: 'help' for help with commands
>         'quit' to quit the shell
>   
>   ><fs> run
>   ><fs> list-devices 
>   /dev/vda
>   ><fs> list-partitions 
>   /dev/vda1
>   /dev/vda2
>   /dev/vda3
>   ><fs> lvs
>   /dev/VolGroup00/LogVol00
>   /dev/VolGroup00/LogVol01
>   ><fs> file /dev/VolGroup00/LogVol00 
>   Linux rev 1.0 ext3 filesystem data (large files)
>   ><fs> mount /dev/VolGroup00/LogVol00 /
>   ><fs> cat /etc/redhat-release 
>   Fedora release 10 (Cambridge)
>   
>   ><fs> vi /etc/passwd
>   ><fs> ^D
> 
> What we do to make this happen is run a small Fedora distro (called
> the "appliance") under qemu which is attached to the disk image in
> question.  The appliance runs a Linux kernel, LVM tools etc etc and
> thus has full access to the disk.  You can also run qemu as non-root,
> which means you don't need to be root to edit disk images.
> 
> The complexity in the specfile is mainly around building this
> appliance.
> 
> We use a tool called febootstrap which is basically a shell script
> that runs 'yum' as non-root, using Debian's fakeroot and fakechroot
> programs.
> 
> febootstrap can take any yum repository and turn it into an appliance,
> without needing root permission:
> 
>   $ febootstrap fedora-11 ./f11
>   $ febootstrap-to-initramfs ./f11 > initrd.img
> 
> Of course during the Koji build we don't have a yum repository, and we
> don't have (or want) network access.  So we make one using createrepo
> + the BuildRequired rpms from the mock cache.
> 
> There's a further complexity here -- a normal appliance would actually
> contain the kernel + glibc + libraries + programs.  However that is
> undesirable from a Fedora point of view, since it's very similar to
> static linking.  If we really shipped that as an appliance then there
> would be obvious security / patching headaches if (for example) glibc
> had a security bug.  Therefore we don't actually bundle any binaries.
> We remove those from the final appliance, and get them from the host
> system at runtime.  The technique is called building a "supermin
> appliance" and it's described here: http://libguestfs.org/README.txt
> under "supermin appliance" It is important to reiterate that the
> appliance we ship in the RPM *does not* contain any libraries or
> binaries.  We just remember where those files were, and pull them out
> of the host's filesystem at runtime.
> 
> From the point of view of improving Koji (or mock) it would be nice to
> have a more official method to access the build RPMs, although the
> technique we are using works fine.

So first question, why isn't the chroot that mock created for you good
enough?  Why do you have to create a second chroot?

> On to the specfile itself:
> 
>   # Enable to build w/o network.
>   %global buildnonet 1
>   
>   Summary:     Access and modify virtual machine disk images
>   Name:        libguestfs
> [...]  
>   # Basic build requirements:
>   BuildRequires: /usr/bin/pod2man
>   BuildRequires: /usr/bin/pod2text
>   BuildRequires: febootstrap >= 2.3
>   BuildRequires: augeas-devel >= 0.5.0
>   BuildRequires: readline-devel
>   BuildRequires: squashfs-tools
>   BuildRequires: qemu-kvm >= 0.10-7
>   BuildRequires: createrepo
>   BuildRequires: glibc-static
>   
>   # This is only needed for RHEL 5 because readline-devel doesn't
>   # properly depend on it, but doesn't do any harm on other platforms:
>   BuildRequires: ncurses-devel
> 
> These are the basic build requires for making the actual package.
>   
>   # Build requirements for the appliance (see 'make.sh.in' in the source):
>   BuildRequires: kernel, bash, coreutils, lvm2, ntfs-3g, util-linux-ng
>   BuildRequires: MAKEDEV, net-tools, augeas-libs, file
>   BuildRequires: module-init-tools, procps, strace, iputils
>   BuildRequires: dosfstools, zerofree, lsof, scrub
>   %ifarch %{ix86} x86_64
>   BuildRequires: grub, ntfsprogs
>   %endif
> 
> These are the BRs reflecting what is inside the appliance ...
> 
>   # Must match the above set of BuildRequires exactly!
>   Requires:      kernel, bash, coreutils, lvm2, ntfs-3g, util-linux-ng
>   Requires:      MAKEDEV, net-tools, augeas-libs, file
>   Requires:      module-init-tools, procps, strace, iputils
>   Requires:      dosfstools, zerofree, lsof, scrub
>   %ifarch %{ix86} x86_64
>   Requires:      grub, ntfsprogs
>   %endif
> 
> ... and at runtime we need the same set of Requires, because of the
> supermin appliance described above.  Those files that we evicted from
> the appliance at build time must exist in the host system at runtime.
> 
> [...]
>   %build
>   %if %{buildnonet}
>   mkdir repo
>   find /var/cache/yum -type f -name '*.rpm' -print0 | xargs -0 cp -t repo
>   createrepo repo
>   %define extra --with-mirror=file://$(pwd)/repo --with-repo=fedora-12 --with-updates=none
>   %else
>   %define extra %nil
>   %endif
> 
> This is the part where we create the local repository from the mock
> cache.  The %{buildnonet} switch allows people to build from the
> Fedora repos instead if they do have network access.
>   
>   # --with-net-if=ne2k_pci is a workaround for RHBZ#516022.
>   ./configure \
>     --prefix=%{_prefix} --libdir=%{_libdir} \
>     --mandir=%{_mandir} \
>     --with-qemu="qemu-kvm qemu-system-%{_build_arch} qemu" \
>     --enable-debug-command \
>     --enable-supermin \
>     --with-net-if=ne2k_pci \
>     %{extra}
>   
>   # This ensures that /usr/sbin/chroot is on the path.  Not needed
>   # except for RHEL 5, it shouldn't do any harm on other platforms.
>   export PATH=/usr/sbin:$PATH
>   
>   # 'INSTALLDIRS' ensures that perl libs are installed in the vendor dir
>   # not the site dir.
>   make INSTALLDIRS=vendor %{?_smp_mflags}
> 
> This bit is building the library + appliance + 6 language bindings.
>   
>   %check
>   # Enable debugging - very useful if a test does fail, although
>   # it produces masses of output in the build.log.
>   export LIBGUESTFS_DEBUG=1
>   
>   # Uncomment one of these, depending on whether you want to
>   # do a very long and thorough test ('make check') or just
>   # a quick test to see if things generally work.
>   
>   # Tracking test issues:
>   # BZ       archs        branch reason
>   # 494075   ppc, ppc64          openbios bug causes "invalid/unsupported opcode"
>   # 504273   ppc, ppc64          "no opcode defined"
>   # 505109   ppc, ppc64          "Boot failure! No secondary bootloader specified"
>   # 502058   i386, x86-64 F-11   need to boot with noapic (WORKAROUND ENABLED)
>   # 502074   i386         F-11   commands segfault randomly
>   # 503236   i386         F-12   cryptomgr_test at doublefault_fn
>   # 507066   all          F-12   sequence of chroot calls (FIXED)
>   # 513249   all          F-12   guestfwd broken in qemu (FIXED)
>   # 516022   all          F-12   virtio-net gives "Network is unreachable" errors
>   #                                 (WORKAROUND ENABLED)
>   # 516096   ?            F-11   race condition in swapoff/blockdev --rereadpt
>   # 516543   ?            F-12   qemu-kvm segfaults when run inside a VM
>   
>   #%ifarch x86_64
>   #make check
>   #%endif
> 
> Normally we run the extensive test suite.  However because of a bug in
> qemu (#516543) we are unable to run that at the moment.
> 
> And the rest is just concerned with the language bindings.
>   
> Rich.
> 

-- 
Jesse Keating
Fedora -- FreedomĀ² is a feature!
identi.ca: http://identi.ca/jkeating

Attachment: signature.asc
Description: This is a digitally signed message part


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]