Issue #4 February 2005

Xen, Virtualization on Linux

For more information on virtualization at Red Hat (and how it's grown since this article was published), see's Virtualization Resource Center. [ Feb. 15, 2007 ]

As some people have noticed, Xen is now available from the Fedora development repository. Xen is a fairly simple operating system that acts as a hypervisor that other OSes can run on top of. Xen is also used to refer to the set of tools for managing the guest operating systems running on top of the Xen hypervisor. Xen is licensed under the GPL. More information on Xen can be found at the Xen virtual machine monitor website.

Xen is a fairly simple operating system that acts as a hypervisor that other OSes can run on top of.

Fedora is following the -unstable Xen tree, which can occasionally lead to things being broken but also lets us track a lot of the more interesting developments in the project. Since setting up to run Xen isn't entirely straight-forward, here's a run-through of what should work for setting up a single Xen guest running the Fedora development tree.

To run Xen on a system pulling strictly from the Fedora development repository, all of your dependencies should be satisfied automatically. Just in case, they are:

  • mkinitrd 4.2.0
  • python 2.4
  • python-twisted
  • Grub as the boot loader
  • 256 MB of RAM with the default setup

Grub is required because you actually boot the Xen hypervisor, and it then starts the Linux kernel. It does this using the MultiBoot standard.

You can conceivably install Xen with less than 256 MB of RAM, but you'll have to reduce the dom0_memory line in /boot/grub/grub.conf. The 130000 for dom0 is currently hard-coded by mkinitrd (which will be fixed soon) and can go a little lower. Also, the Xen hypervisor ends up requiring around 32 MB plus any memory for your unprivileged guests.

You should be able to install the xen and kernel-xen0 packages. Once this is done, you should have an entry set up in your grub.conf to boot the xen0 kernel. Now, reboot into your new xen0 kernel.

You may need to disable rhgb and switch to using runlevel 3. There are conflicting reports about X working; I haven't tried.

Once you've rebooted, you should be running in the dom0 kernel. You'll see a slightly scary looking warning about TLS during bootup and how to disable it, but it shouldn't make things too bad. If you start xend with the command service xend start, you should be able to run xm list and see your domain running. Now, we want to set up a simple base Fedora system. First, you'll want to install the kernel-xenU package (unfortunately, the kernel for your guest domain must currently be kept outside of the guest itself).

Next, let's create a file to use as the backing for our Fedora install. For example purposes, I'll create one of a size of 1 GB.

dd if=/dev/zero of=/root/fedora.img bs=1M count=1 seek=1024

Now, create an ext3 filesystem on this image.

mke2fs -F -j /root/fedora.img

You should now be able to mount your new temporary rootfs on a temporary mountpoint such as /mnt:

mount -o loop /root/fedora.img /mnt

Now, install whatever base system you want into this chroot. Make sure that your yum configuration points to a valid repository. Then, decide what group(s) you want to install. I'd recommend starting with Base (or for the space constrained, Core, but this is more difficult). Then, execute:

yum --installroot=/mnt -y groupinstall Base

Now, go get some coffee and have a snack. It's going to take a little while. :-)

Come back and if everything went okay, you'll have a minimal install in /mnt/.

Now, for the ugly part: we need to set up some basic bits on the filesystem that have to be different for Xen right now. These include a) creating some required device nodes in /dev since we're not using an initrd and b) setting up an /etc/fstab file.

for i in console null zero ; do MAKEDEV -d /mnt/dev -x $i ; done

For the /etc/fstab, something simple like the following should work:

/dev/sda1               /                       ext3    defaults 1 1
none                    /dev/pts                devpts  gid=5,mode=620 0 0
none                    /dev/shm                tmpfs   defaults 0 0
none                    /proc                   proc    defaults 0 0
none                    /sys                    sysfs   defaults 0 0

Do any other configuration you want to on the filesystem and then unmount it and /proc.

umount /mnt{/proc,}

Now, create a configuration file and you should be good to go. I like a very simple config file like /etc/xen/rawhide on my machine.

Substitute /boot/vmlinuz-2.6.10-1.1090_FC4xenU with the xenU kernel you installed. Additionally, the device sda1 listed here as the disk is related to the /etc/fstab you created previously.

kernel ="/boot/vmlinuz-2.6.10-1.1103_FC4xenU"
memory = 64
name = "rawhide"
nics = 1
disk = ['file:/root/fedora.img,sda1,w']
root = "/dev/sda1 ro"

Now, create a new domain:

xm create -c rawhide

At the end, you should see the login prompt. At this point you can login as root and begin playing around.

This is pretty early and rough, but it's enough to started. The next step (for me :)) is getting to where you can do actual installs in a Xen guest environment and boot kernels which are on the guest's filesystem.

The guest operating system is aware it is running on the Xen virtual machine, so certain parts of it need to be ported accordingly. One advantage is that it can be optimized for the VM. Additionally, for guest operating systems that aren't running in domain 0 (which we can get to), you have to have support for the "devices" given to the guests. For example, network devices show up as ethX but use a different drivers (xen_net instead of e1000, for example). Operating systems that currently run as Xen guests include Linux, FreeBSD (in progress), netBSD, and Plan 9. Currently, it only runs on the x86 architecture, although work is underway on both the x86_64 and ia64 ports.

How does Xen compare to similar VMs such as VMWare and user-mode Linux? Some VMWare users will find it beneficial, but others will not. VMWare provides a full virtualized machine that looks, smells, and tastes like a "real" PC— it even has a BIOS. Xen is paravirtualized, hence the necessary changes. (Paravirtualized means you have to do some porting and can't just run an existing OS unmodified.) Xen more likely to be a replacement for people using user-mode Linux now.

Work to integrate Xen into Fedora Core is ongoing. If you want to jump in and help, testing and feedback is always welcome. Specific failures should be reported to Bugzilla. Otherwise, fedora-devel-list is a good place to ask questions or give feedback. There's a lot of mail on the mailing list, but I do monitor it because it's incredibly important to keep up with it to foster community participation in Fedora.

For Fedora Core 4, we are trying to get Xen as fully integrated as possible. Even if we don't get it all done, it should be to a state where people interested in the technology can start using it and get a feel for the capabilities and performance. Additional feedback based on the early integration will help guide future efforts around the technology.

Editor's Note:
Jeremy will be at FUDCon discussing virtualization, among other topics.

About the author

Jeremy Katz is helping to lead Red Hat's internal investigation and work on integrating Xen. He is also the leader of the installer team.