[Ovirt-devel] Ovirt Host Tasks
Perry N. Myers
pmyers at redhat.com
Mon Mar 17 03:43:02 UTC 2008
Here's an initial cut at an Ovirt Host task list. After we refine this a
little with some discussion, I'll prioritize these.
One thing that complicates the design is supporting static addressing of
Ovirt hosts. Making the assumption that DHCP can be used and configured
for Ovirt makes things much simpler. I won't make the DHCP assumption
below, but would appreciate people's thoughts on whether or not static
addressing support is important.
1. Image Creation:
Right now the Ovirt Host is created using scripts in the Ovirt host
repository that utilize livecd creator to make ISO, USB and PXE formats.
I think it makes sense to start with the tools that Chris L. has already
been working with and migrate new functionality in, rather than starting
with other tools.
a. Integrate whitelist functionality into the image creation
process to reduce image size.
b. Determine the minimal set of files that the host image needs to
construct a baseline whitelist.
c. Determine minimal set of kernel modules to support wide platform base
(i.e. keep disk, net modules. remove things like sound, isdn, etc...)
d. Construct a repeatable build process so that images can be built
daily for testing.
e. Construct a utility that detects the minimal set of kernel modules
for a given platform and generates a config file that can be fed to
the build system for generating hardware platform specific images.
f. Construct a method for taking the generic host image and removing all
kernel modules not applicable for a specific platform.
g. Construct a method for taking an image and adding site specific
configuration information to it (IP addresses of Ovirt Mgmt Server and
FreeIPA Server). This is only necessary to support hosts that require
static IP configuration -and- do not have persistent storage to store
configuration information. (If we decide to support static IP config
we could make presence of persistent storage a requirement...)
The goal is to get disk footprint down to something like 32, 64 or 128MB.
The base images we build will not be hardware platform specific, so 32MB
might be a long shot. But after step e) above, we should fit into 32MB.
2. Installation
We have to account for two methods of distributing Ovirt: pre-installation
by OEMs and install by IT admins. Pre-install by OEM implies that the
platform has flash or a reserved partition on the primary hard disk.
Install by IT can be done by setting up PXE boot, adding an external
bootable USB disk, booting to CD or installing to onboard flash/disk.
The livecd/liveusb disk should allow either booting and running directly
or installation onto platform flash/disk.
3. Persistent Configuration Storage
Since the host is non-persistent and may or may not have access to
persistent storage, this could become tricky. It's further complicated by
the need to support DHCP and static addressing of the host.
The following things need to be stored for the host:
a. Kerberos keytab or SSL Certificate
b. IP configuration (if using static configuration)
c. IP address of FreeIPA server and Ovirt Management Server (if using
static configuration)
Here's where things could be much simpler if DHCP were required. Without
static addressing, b is not necessary and c can be transmitted as custom
DHCP fields (iirc). Then the only thing that needs to be cached on the
host on persistent storage is the auth credentials. (NOTE: Does
Microsoft's DHCP server allow custom DHCP fields???)
OEM Pre-Install (will always have flash/disk storage)
DHCP Addressing
Flash/Disk Storage/TPM
Auth credentials can be stored in a file or in TPM
Static Addressing
Flash/Disk Storage/TPM
Auth credentials/IP config/Server IPs can be stored in a file
If TPM is present, can be used to store auth info
IT Install (may not have flash or disk)
DHCP Addressing
Flash/Disk Storage/TPM
Auth credentials can be stored in a file or in TPM
No Persistent Storage (CD or PXE Boot)
Auth can be stored in host image (separate image for each host)
(NOTE: Not secure for PXE boot. Recommend that we only allow
CD boot in this case)
Static Addressing
Flash/Disk Storage/TPM
Auth credentials/IP config/Server IPs can be stored in a file
If TPM is present, can be used to store auth info
No Persistent Storage (CD or PXE Boot)
Auth/IP config/Server IPs can be stored in host image (separate
image for each host)
(NOTE: Not secure for PXE boot as above)
4. Initial Configuration
When the host comes up the first time it needs to know the IP address and
password of the Ovirt Mgmt server so it can get its auth credentials set
up. Here's how a typical setup might look:
a. Cable up the new hardware and get MAC addrs for each iface. Need
to record which MAC is for:
* management network (NOTE: should be only iface that PXE boots
if PXE is being used)
* storage network
* normal networks
(these all don't have to be separate, there could be just a single
interface on the box used for management, storage and normal traffic)
b. Set up DNS/DHCP servers. If using static addressing, skip DHCP setup
(This is a manual step that IT admin would do, i.e. we're not trying
to automate this process)
c. Boot Ovirt Host for first time. Kernel cmdline takes options like:
* ovirt_net=static (to indicate that static ip config should be
prompted for during boot)
* ovirt_auth_pw=<password> (password to use when connecting to the
Ovirt Mgmt server to register the host and retrieve auth information)
* ovirt_init (this is specified to let the host know that initial
config needs to happen. Otherwise normal boot occurs)
d. During boot if ovirt_net=static is specified, admin is prompted for
IP config for each interface and which interface is the mgmt interface.
This can be done with an anaconda style interface.
e. Auth Setup:
* If ovirt_auth_pw is not specified: Auth information is expected to be
either on the host disk image or on a connected USB key. (could be
that the Ovirt host image is on internal flash/disk and admin
supplies auth info on a USB thumb drive)
* If ovirt_auth_pw is specified: After network comes up, mgmt server is
contacted and the host is registered using the supplied password.
The auth information is sent back to the Ovirt Host.
f. The host is checked for persistent storage and TPM devices. If TPM
is found it is initialized so that auth info can be stored there.
* If either USB or hard disk is found it is checked for a partition
with a label called "OVIRT". If that exists it is used for
persistent storage. If it does not, an "OVIRT" partition is created.
on available unpartitioned space. (how big should it be?)
* USB/disk is also checked for swap partition. If none is found, a
swap partition is created using unpartitioned space.
g. Host stores auth/ip information on persistent storage/TPM for future
boots.
5. KVM Work
* Need to work on support for Live Migration
* Need to make sure that power management and suspend/resume of guests
work as expected
6. Storage Support
* libvirt storage API to support fibre channel
* Work on using LVM on top of iSCSI so that we don't need to have a 1-1
mapping between LUNs and Guests
* libvirt already supports NFS, just need to put plumbing in place so that
Ovirt also supports NFS. (This is more of an issue for the management
interface, not the Ovirt Host)
7. Performance Monitoring/Auditing/Health
* collectd is used presently to send performance stats to management
server. Is this the right solution? I don't have a better one, but
suggestions are appreciated.
* libvirt talks to auditd to send audit info to FreeIPA server
* Need to create a health monitoring daemon that communicates to mgmt
server:
- Sends heartbeat at configurable interval
- Sends status changes of host (including machine check exceptions,
and other errors)
- Sends VM state changes
8. Clustering
I'm light on clustering knowledge, so fill me in if I'm talking crazy
nonsense...
Two types of clustering: physical machines and virtual machines
Physical clustering is not particularly useful in and of itself, since the
whole point of what we're trying to do is abstract the applications from
hardware further.
Clustering of guests should be done. The hardware fence would be replaced
by a paravirt driver in the guest that communicates to the host. If two
machines A and B are clustered and on different physical hardware, A is
the primary and B is backup. B monitors the application on A. If A
appears to go down, B uses the PV driver to tell the host that it needs to
kill A. The host tells libvirt that A needs to be destroyed, and that
command is sent over libvirt comms to the physical host that is hosting A.
If the physical hardware hosting A is down, that libvirt command won't
succeed. So in this case, a hardware fence would be required to make sure
that the physical host is disconnected and does not power on again.
This probably needs a lot of refinement. So someone who knows clustering
better than I do, should feel free to help out here...
9. Image Updates
Host images should be updated as binary blobs. (initially a complete
overwrite, and eventually perhaps a binary diff)
Updating can be done in a few ways:
a. For machines that PXE, this is simply putting a new PXE image in place.
b. For machines that boot from CD a new livecd needs to be created and
physically put in machines
c. For machines that boot from USB/hard disk an update daemon can run on
the host that polls the mgmt server for new images. When a new image
is found it is retrieved and stored on the persistent disk and grub
is updated to boot to it. If boot fails of the new image, the next
boot falls back to the old image. (This means that persistent storage
should be 2 times the size of the compressed image size + some
additional room for config)
That's a mouthful. Thoughts and criticisms appreciated.
Perry
--
|=- Red Hat, Engineering, Emerging Technologies, Boston -=|
|=- Email: pmyers at redhat.com -=|
|=- Office: +1 412 474 3552 Mobile: +1 703 362 9622 -=|
|=- GnuPG: E65E4F3D 88F9 F1C9 C2F3 1303 01FE 817C C5D2 8B91 E65E 4F3D -=|
More information about the ovirt-devel
mailing list