[Fedora-livecd-list] PATCH: Disk (appliance) creator tool

Daniel P. Berrange berrange at redhat.com
Tue Feb 19 19:16:51 UTC 2008


On Tue, Feb 19, 2008 at 01:58:46PM -0500, Jeremy Katz wrote:
> On Sun, 2008-02-17 at 02:14 +0000, Daniel P. Berrange wrote:
> > This patch adds support for another installation variant. The livecd-creator
> > produces ISO images which boot using syslinux. The image-creator creates a
> > single file containing a filesystem in which the OS is installated. This patch
> > adds a new disk-creator, which creates a single file containing a partitioned
> > disk with potentially many filesystems in which the OS is installed. Furthermore
> > it installs grub in the boot sector, so this image is immediately bootable in
> > a virtual machine.
> 
> Seems reasonable enough and it's something I had started looking at.  My
> biggest concerns end up being scope... the next thing someone will want
> to add is RAID.  Or LVM.  Or dm-crypt.  And I sort of wonder where it
> ends.  I already gets bugs on one copy of code like that and it's called
> anaconda ;-)

Where to draw the line is an interesting question. If you consider the created
appliance as primary serving for deployment in virtual machines, then I'd
argue that RAID / crypt is out of scope. They would be a site-specific deployment
question. If an admin wants encryption they can set it up in the host directly.
Likewise RAID can be done in the host already so its redundant for guests.

LVM would be useful if only to make adding storage to the appliance easier.

So I'd declare that scope is limited to plain partitions + LVM.

> > I've used this tool to create an instance of the oVirt web management appliance
> > from its kickstart recipe.  Currently developers building the appliance have to
> > boot a VM using the F8 boot.iso and run the kickstart script in the VM and so
> > on. While this works, involves many steps with potential for failure. This new
> > tool reduces the problem to simply
> 
> As opposed to virt-install --name ... ?  I'm not convinced there's a
> huge gain in terms of number of places for failure :-/

You have to have a working virtualization stack & it takes more resource
overhead than just doing the chroot'd build. Emprically the part of our
build process doing the guest based installs has been less reliable than
the livecd-creator part. Havinga  disk-creator will address that problem

> > With the background information out of the way, here's some info about the
> > changes in this patch...
> 
> Ob-side-note: your patch doesn't seem to be against master or at least,
> is missing some bits as it doesn't want to apply.  

I used a fresh git checkout from saturday for doing all the work against
and don't see any changes since then.

> > Most of the change here involves re-factoring the imgcreate/fs.py module.
> > The SparseExtLoopbackMount, SparseLoopbackMount, LoopbackMount classes all
> > have the built-in limitation that the image being produced corresponds to
> > a single filesystem / loop mount.  With the disk creator, the image being
> > produced can be partitioned into multiple chunks, with many filesystems.
> > Furthermore, not all of them require loopback mounts, as the partitions
> > themselves are already visible via /dev/mapper/
> > 
> > So this patch separates the roles. There are now classes which deal with
> > accessing / creating disks:
> [snip]
> > The 'create' method must make the disk visible as a block device - eg
> > by calling losetup. For RawDisk, this is obviously a no-op. The 'cleanup'
> > method must undo the 'create' operation.
> 
> Sounds okay...  the API change involved is a little unfortunate, but I
> did caveat that the API in -14 isn't guaranteed, so it should be fine.

If its really a problem, we've got the option of just leaving the existing
classes unchanged & deprecating their use in favour of the new ones, but
if no one is using the API seriously yet it might not be worth the hassle.

> > Next, up we have a new class 'PartitionedMount'. This takes a Disk object
> > and adds one or more partitions to it. It then creates further Disk and
> > Mount object instances for each partition. So, mounting a 'PartitionedMount'
> > object will in fact mount all its partitions (formatting the filesystems
> > or swap space as needed).
> 
> It looks like you're calling parted directly here rather than using the
> python bindings?  The latter is probably somewhat preferable.  Also,
> there are probably some interesting questions about always assuming that
> you want an msdos partition table :-/  This goes back to my first
> point...

I didn't realize there were useful parted python bindings. I'll happily
use them if that's desired.  

The use of msdos partitions can easily be addressed - I just didn't look
to see if there's a kickstart syntax option for specifying partition table
type yet.

> [snip]
> > It doesn't currently care about --ondisk only outputting a single disk
> > image, but I intend to extend the CLI to allow creation of installs which
> > spawn multiple output disks.
> 
> My only concern with doing it on the CLI is that it makes
> reproducability of the images more difficult.  And then what happens if
> my config references two disks but I only gave one on the command line
> or other permutations.  I wonder if we want to come up with some syntax
> like  (warning: completely made up and not really thought about much at
> all yet :-)
>    disk DISKA foo.img
>    disk DISKB bar.img
> 
> That might tie nicely with giving people an alternate way of defining
> their disks on a real kickstart too rather than depending on kernel
> naming.  More thought required...

To be honest we could simply name the disks according to the device names
given in the kickstart file - eg    ovirt-wui-sda.raw & ovirt-wui-sdb.raw.

If we want to mandate the specification of '--size' flag for all partitions
in the kickstart and not support '--grow', then we don't need any CLI flags
for disk-creator at all. This would certainly aid reproducability

> > Aside from partitioning, DiskImageCreator will setup an fstab file containing
> > entries for all the filesystems and swap space. It will add the grub device
> > map file, and create a grub.cfg based off the installed kernel. It is able
> > to cope with a layout where grub config is on /, or a separate /boot partition.
> > We can't use grub-install since it doesn't understand loopdevices, so to do
> > the MBR install we invoke grub manually.
> 
> Yeah, and this can be tricky.  At least historically, there have been
> cases which setup wasn't sufficient for and we had to do the full
> install invocation.  Which is why that's what anaconda does.

On the flip side anaconda has a much broader use case than we have to worry
about for creating appliances. Looking at the final grub commands that
grub-install actually runs, there's nothing magic we can't do directly. The
problem with grub-install is all the 'sanity' checking & probing it does 
before actually running grub, which simply doesn't apply to loop devices.

Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 




More information about the Fedora-livecd-list mailing list