[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Ovirt-devel] Ovirt Host Tasks

Hugh O. Brock wrote:
On Sun, Mar 16, 2008 at 11:43:02PM -0400, Perry N. Myers wrote:

Wow that's a big mail <g>.

I have a couple of general comments based on the way we *have been*
thinking about this stuff (this of course doesn't mean we can't
First off, we have been imagining the oVirt host image as stateless
except for whatever we use to store its identity (if we even do
that). This means we simply don't support static addressing of oVirt
hosts, at all. The appealing thing about a stateless oVirt host image
to me is that it means the host is immediately and easily upgradeable;
there is 0 local state to worry about aside from the Kerberos
keytab. It also means that a reboot will immediately restore a
compromised machine FWIW.
Having said all that, if there is a compelling reason to move away
from the stateless idea I'm all ears.

I'm all for getting rid of static config support. The fundamental question is, how many people would require static IP config (as opposed to static DHCP reservations)? If that is a significant number of potential users, it makes sense to support it.

We can always take the approach of: implement stateless/DHCP support only and then if we get feedback from enough folks that static addressing is important enough we can try to add it in later.

Unless I hear major objections, this is how I'll proceed.

Further comments in-line below...
1. Image Creation:

Right now the Ovirt Host is created using scripts in the Ovirt host repository that utilize livecd creator to make ISO, USB and PXE formats.

I think it makes sense to start with the tools that Chris L. has already been working with and migrate new functionality in, rather than starting with other tools.

a. Integrate whitelist functionality into the image creation
   process to reduce image size.
b. Determine the minimal set of files that the host image needs to
   construct a baseline whitelist.
c. Determine minimal set of kernel modules to support wide platform base
   (i.e. keep disk, net modules.  remove things like sound, isdn, etc...)
d. Construct a repeatable build process so that images can be built
   daily for testing.
e. Construct a utility that detects the minimal set of kernel modules
   for a given platform and generates a config file that can be fed to
   the build system for generating hardware platform specific images.
f. Construct a method for taking the generic host image and removing all
   kernel modules not applicable for a specific platform.
g. Construct a method for taking an image and adding site specific
   configuration information to it (IP addresses of Ovirt Mgmt Server and
   FreeIPA Server).  This is only necessary to support hosts that require
   static IP configuration -and- do not have persistent storage to store
   configuration information.  (If we decide to support static IP config
   we could make presence of persistent storage a requirement...)

You could add "Kerberos keytab" to "site specific configuration
information" and the host image would be completely
self-contained. I can imagine a library of unique images under

I don't like the idea of having a separate PXE image for each individual host unless -absolutely- necessary. So we shouldn't encourage this.

So here are some assumptions we would should make:

* DHCP only
* keytab/SSL cert stored in TPM if available
* If TPM not available, keytab/SSL cert stored on local storage (USB/disk)
* If no persistent storage available, just attach the smallest thumb drive
  that you have to the box. (i.e. *make* persistent storage available)

2. Installation

We have to account for two methods of distributing Ovirt: pre-installation by OEMs and install by IT admins. Pre-install by OEM implies that the platform has flash or a reserved partition on the primary hard disk. Install by IT can be done by setting up PXE boot, adding an external bootable USB disk, booting to CD or installing to onboard flash/disk.

The livecd/liveusb disk should allow either booting and running directly or installation onto platform flash/disk.

That's a nice addition, yeah. Although again I'm in favor of the
running image itself remaining stateless.

The image is still stateless in this scenario. It's just transferring the boot image from removable media to onboard media. Once it is transferred, you remove the usb/cd media.

3. Persistent Configuration Storage

Since the host is non-persistent and may or may not have access to persistent storage, this could become tricky. It's further complicated by the need to support DHCP and static addressing of the host.

The following things need to be stored for the host:
a. Kerberos keytab or SSL Certificate
b. IP configuration (if using static configuration)
c. IP address of FreeIPA server and Ovirt Management Server (if using
   static configuration)

Here's where things could be much simpler if DHCP were required. Without static addressing, b is not necessary and c can be transmitted as custom DHCP fields (iirc). Then the only thing that needs to be cached on the host on persistent storage is the auth credentials. (NOTE: Does Microsoft's DHCP server allow custom DHCP fields???)

Actually we've talked about moving away from overloading DHCP for this
stuff and instead handling it with mdns (let zeroconf be

Echoing conversations on the IRC channel...

zeroconf/avahi is not reliable, so even on a small scale it is probably not a good idea. dhcp fields work fine, but may not play well with MS DHCP servers.

So in the short term, we support sending out critical config info (ip addresses of certain servers) via dhcp fields.

For environments where we can't add custom dhcp fields, perhaps we should set up well known host aliases. i.e. if the dhcp fields are not present we assume that the freeipa server is located at freeipa.domain.com (where domain.com is the domain specified in dhcp) and the ovirt mgmt server is ovirt-mgmt.domain.com.

This would mean that DNS records would need to be set up for these two servers, but we're already asking people to muck with DNS so....

There is another interesting possibility we might want to look at in the
medium term. We are actively working on ways to use amqp as part of
our infrastructure. If a machine has an identity via a keytab and an
IP address via DHCP, I imagine it's possible to retrieve the remaining
necessary config information by sending a message to a known alias?
This relieves the oVirt host of having to know the precise address of
the management server. Something to think about at least.

Interesting idea.

4. Initial Configuration

When the host comes up the first time it needs to know the IP address and password of the Ovirt Mgmt server so it can get its auth credentials set up. Here's how a typical setup might look:

a. Cable up the new hardware and get MAC addrs for each iface.  Need
   to record which MAC is for:
   * management network (NOTE: should be only iface that PXE boots
     if PXE is being used)
   * storage network
   * normal networks
   (these all don't have to be separate, there could be just a single
   interface on the box used for management, storage and normal traffic)
b. Set up DNS/DHCP servers.  If using static addressing, skip DHCP setup
   (This is a manual step that IT admin would do, i.e. we're not trying
   to automate this process)
c. Boot Ovirt Host for first time.  Kernel cmdline takes options like:
   * ovirt_net=static (to indicate that static ip config should be
     prompted for during boot)
   * ovirt_auth_pw=<password> (password to use when connecting to the
     Ovirt Mgmt server to register the host and retrieve auth information)
   * ovirt_init (this is specified to let the host know that initial
     config needs to happen.  Otherwise normal boot occurs)

Getting back to the stateless idea, I think the host image should
always boot exactly the same way and be smart enough to know whether
it needs config information or not and, if so, ask for it? Having said
that the "password" idea is reasonable except that you would always
need to be there to enter the password for a reboot...

The password is only used to register the host on firstboot and to securely retrieve the keytab/SSL Cert from the management server.

So, if as above we eliminate the need for static IP config, we can reduce the cmdline options to:

* ovirt_auth=<password>
* ovirt_auth=floppy:/keytab
* ovirt_auth=hd:sdb1:/keytab

So either you specify a password to contact the ovirt host for registration and keytab retrieval or you specify the location of the keytab on a floppy/usb thumbdrive (syntax similar to kickstart cmdline syntax)

f. The host is checked for persistent storage and TPM devices.  If TPM
   is found it is initialized so that auth info can be stored there.
   * If either USB or hard disk is found it is checked for a partition
     with a label called "OVIRT".  If that exists it is used for
     persistent storage.  If it does not, an "OVIRT" partition is created.
     on available unpartitioned space. (how big should it be?)
   * USB/disk is also checked for swap partition.  If none is found, a
     swap partition is created using unpartitioned space.

Hadn't thought about using TPM for the keytab, that's a neat idea. I'm
a bit leery of using any local HDD on the machine for it though.

Why?  Are you ok with using USB storage but not ok with using HDD storage?


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]