[fedora-virt] Re: Spec weirdness
Jesse Keating
jkeating at redhat.com
Wed Aug 12 23:56:15 UTC 2009
On Mon, 2009-08-10 at 19:45 +0100, Richard W.M. Jones wrote:
> On Mon, Aug 10, 2009 at 10:40:16AM -0700, Jesse Keating wrote:
> > I'm looking at the spec file for libguestfs, and all I can say is WTF.
> > There is a lot of crazyness going on in this spec, chroots within
> > chroots, making a repo of yum cache packages and using it again, calling
> > qemu, and none of it is really documented in the spec as for what and
> > why things are done this way.
>
> Hey Jesse, I'll explain below what the general idea is, then provide
> at the end some commentary inline on specific points within the
> specfile. It certainly is a complex specfile.
>
> In case you aren't familiar with it, libguestfs (http://libguestfs.org)
> is a tool for modifying VM disk images. For example:
>
> $ guestfish -a Fedora10.img
>
> Welcome to guestfish, the libguestfs filesystem interactive shell for
> editing virtual machine filesystems.
>
> Type: 'help' for help with commands
> 'quit' to quit the shell
>
> ><fs> run
> ><fs> list-devices
> /dev/vda
> ><fs> list-partitions
> /dev/vda1
> /dev/vda2
> /dev/vda3
> ><fs> lvs
> /dev/VolGroup00/LogVol00
> /dev/VolGroup00/LogVol01
> ><fs> file /dev/VolGroup00/LogVol00
> Linux rev 1.0 ext3 filesystem data (large files)
> ><fs> mount /dev/VolGroup00/LogVol00 /
> ><fs> cat /etc/redhat-release
> Fedora release 10 (Cambridge)
>
> ><fs> vi /etc/passwd
> ><fs> ^D
>
> What we do to make this happen is run a small Fedora distro (called
> the "appliance") under qemu which is attached to the disk image in
> question. The appliance runs a Linux kernel, LVM tools etc etc and
> thus has full access to the disk. You can also run qemu as non-root,
> which means you don't need to be root to edit disk images.
>
> The complexity in the specfile is mainly around building this
> appliance.
>
> We use a tool called febootstrap which is basically a shell script
> that runs 'yum' as non-root, using Debian's fakeroot and fakechroot
> programs.
>
> febootstrap can take any yum repository and turn it into an appliance,
> without needing root permission:
>
> $ febootstrap fedora-11 ./f11
> $ febootstrap-to-initramfs ./f11 > initrd.img
>
> Of course during the Koji build we don't have a yum repository, and we
> don't have (or want) network access. So we make one using createrepo
> + the BuildRequired rpms from the mock cache.
>
> There's a further complexity here -- a normal appliance would actually
> contain the kernel + glibc + libraries + programs. However that is
> undesirable from a Fedora point of view, since it's very similar to
> static linking. If we really shipped that as an appliance then there
> would be obvious security / patching headaches if (for example) glibc
> had a security bug. Therefore we don't actually bundle any binaries.
> We remove those from the final appliance, and get them from the host
> system at runtime. The technique is called building a "supermin
> appliance" and it's described here: http://libguestfs.org/README.txt
> under "supermin appliance" It is important to reiterate that the
> appliance we ship in the RPM *does not* contain any libraries or
> binaries. We just remember where those files were, and pull them out
> of the host's filesystem at runtime.
>
> From the point of view of improving Koji (or mock) it would be nice to
> have a more official method to access the build RPMs, although the
> technique we are using works fine.
So first question, why isn't the chroot that mock created for you good
enough? Why do you have to create a second chroot?
> On to the specfile itself:
>
> # Enable to build w/o network.
> %global buildnonet 1
>
> Summary: Access and modify virtual machine disk images
> Name: libguestfs
> [...]
> # Basic build requirements:
> BuildRequires: /usr/bin/pod2man
> BuildRequires: /usr/bin/pod2text
> BuildRequires: febootstrap >= 2.3
> BuildRequires: augeas-devel >= 0.5.0
> BuildRequires: readline-devel
> BuildRequires: squashfs-tools
> BuildRequires: qemu-kvm >= 0.10-7
> BuildRequires: createrepo
> BuildRequires: glibc-static
>
> # This is only needed for RHEL 5 because readline-devel doesn't
> # properly depend on it, but doesn't do any harm on other platforms:
> BuildRequires: ncurses-devel
>
> These are the basic build requires for making the actual package.
>
> # Build requirements for the appliance (see 'make.sh.in' in the source):
> BuildRequires: kernel, bash, coreutils, lvm2, ntfs-3g, util-linux-ng
> BuildRequires: MAKEDEV, net-tools, augeas-libs, file
> BuildRequires: module-init-tools, procps, strace, iputils
> BuildRequires: dosfstools, zerofree, lsof, scrub
> %ifarch %{ix86} x86_64
> BuildRequires: grub, ntfsprogs
> %endif
>
> These are the BRs reflecting what is inside the appliance ...
>
> # Must match the above set of BuildRequires exactly!
> Requires: kernel, bash, coreutils, lvm2, ntfs-3g, util-linux-ng
> Requires: MAKEDEV, net-tools, augeas-libs, file
> Requires: module-init-tools, procps, strace, iputils
> Requires: dosfstools, zerofree, lsof, scrub
> %ifarch %{ix86} x86_64
> Requires: grub, ntfsprogs
> %endif
>
> ... and at runtime we need the same set of Requires, because of the
> supermin appliance described above. Those files that we evicted from
> the appliance at build time must exist in the host system at runtime.
>
> [...]
> %build
> %if %{buildnonet}
> mkdir repo
> find /var/cache/yum -type f -name '*.rpm' -print0 | xargs -0 cp -t repo
> createrepo repo
> %define extra --with-mirror=file://$(pwd)/repo --with-repo=fedora-12 --with-updates=none
> %else
> %define extra %nil
> %endif
>
> This is the part where we create the local repository from the mock
> cache. The %{buildnonet} switch allows people to build from the
> Fedora repos instead if they do have network access.
>
> # --with-net-if=ne2k_pci is a workaround for RHBZ#516022.
> ./configure \
> --prefix=%{_prefix} --libdir=%{_libdir} \
> --mandir=%{_mandir} \
> --with-qemu="qemu-kvm qemu-system-%{_build_arch} qemu" \
> --enable-debug-command \
> --enable-supermin \
> --with-net-if=ne2k_pci \
> %{extra}
>
> # This ensures that /usr/sbin/chroot is on the path. Not needed
> # except for RHEL 5, it shouldn't do any harm on other platforms.
> export PATH=/usr/sbin:$PATH
>
> # 'INSTALLDIRS' ensures that perl libs are installed in the vendor dir
> # not the site dir.
> make INSTALLDIRS=vendor %{?_smp_mflags}
>
> This bit is building the library + appliance + 6 language bindings.
>
> %check
> # Enable debugging - very useful if a test does fail, although
> # it produces masses of output in the build.log.
> export LIBGUESTFS_DEBUG=1
>
> # Uncomment one of these, depending on whether you want to
> # do a very long and thorough test ('make check') or just
> # a quick test to see if things generally work.
>
> # Tracking test issues:
> # BZ archs branch reason
> # 494075 ppc, ppc64 openbios bug causes "invalid/unsupported opcode"
> # 504273 ppc, ppc64 "no opcode defined"
> # 505109 ppc, ppc64 "Boot failure! No secondary bootloader specified"
> # 502058 i386, x86-64 F-11 need to boot with noapic (WORKAROUND ENABLED)
> # 502074 i386 F-11 commands segfault randomly
> # 503236 i386 F-12 cryptomgr_test at doublefault_fn
> # 507066 all F-12 sequence of chroot calls (FIXED)
> # 513249 all F-12 guestfwd broken in qemu (FIXED)
> # 516022 all F-12 virtio-net gives "Network is unreachable" errors
> # (WORKAROUND ENABLED)
> # 516096 ? F-11 race condition in swapoff/blockdev --rereadpt
> # 516543 ? F-12 qemu-kvm segfaults when run inside a VM
>
> #%ifarch x86_64
> #make check
> #%endif
>
> Normally we run the extensive test suite. However because of a bug in
> qemu (#516543) we are unable to run that at the moment.
>
> And the rest is just concerned with the language bindings.
>
> Rich.
>
--
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://listman.redhat.com/archives/fedora-virt/attachments/20090812/e6730068/attachment.sig>
More information about the Fedora-virt
mailing list