[Ovirt-devel] Ovirt Host Tasks

[Perry N. Myers]

Hugh O. Brock wrote:
> On Sun, Mar 16, 2008 at 11:43:02PM -0400, Perry N. Myers wrote:
> 
> Wow that's a big mail <g>.
> 
> I have a couple of general comments based on the way we *have been*
> thinking about this stuff (this of course doesn't mean we can't
> change). 
> 
> First off, we have been imagining the oVirt host image as stateless
> except for whatever we use to store its identity (if we even do
> that). This means we simply don't support static addressing of oVirt
> hosts, at all. The appealing thing about a stateless oVirt host image
> to me is that it means the host is immediately and easily upgradeable;
> there is 0 local state to worry about aside from the Kerberos
> keytab. It also means that a reboot will immediately restore a
> compromised machine FWIW. 
> 
> Having said all that, if there is a compelling reason to move away
> from the stateless idea I'm all ears.

I'm all for getting rid of static config support.  The fundamental 
question is, how many people would require static IP config (as opposed to 
static DHCP reservations)?  If that is a significant number of potential 
users, it makes sense to support it.

We can always take the approach of: implement stateless/DHCP support only 
and then if we get feedback from enough folks that static addressing is 
important enough we can try to add it in later.

Unless I hear major objections, this is how I'll proceed.

> Further comments in-line below...
>> 1. Image Creation:
>>
>> Right now the Ovirt Host is created using scripts in the Ovirt host 
>> repository that utilize livecd creator to make ISO, USB and PXE formats.
>>
>> I think it makes sense to start with the tools that Chris L. has already 
>> been working with and migrate new functionality in, rather than starting 
>> with other tools.
>>
>> a. Integrate whitelist functionality into the image creation
>>    process to reduce image size.
>> b. Determine the minimal set of files that the host image needs to
>>    construct a baseline whitelist.
>> c. Determine minimal set of kernel modules to support wide platform base
>>    (i.e. keep disk, net modules.  remove things like sound, isdn, etc...)
>> d. Construct a repeatable build process so that images can be built
>>    daily for testing.
>> e. Construct a utility that detects the minimal set of kernel modules
>>    for a given platform and generates a config file that can be fed to
>>    the build system for generating hardware platform specific images.
>> f. Construct a method for taking the generic host image and removing all
>>    kernel modules not applicable for a specific platform.
>> g. Construct a method for taking an image and adding site specific
>>    configuration information to it (IP addresses of Ovirt Mgmt Server and
>>    FreeIPA Server).  This is only necessary to support hosts that require
>>    static IP configuration -and- do not have persistent storage to store
>>    configuration information.  (If we decide to support static IP config
>>    we could make presence of persistent storage a requirement...)
> 
> You could add "Kerberos keytab" to "site specific configuration
> information" and the host image would be completely
> self-contained. I can imagine a library of unique images under
> /tftpboot...?

I don't like the idea of having a separate PXE image for each individual 
host unless -absolutely- necessary.  So we shouldn't encourage this.

So here are some assumptions we would should make:

* DHCP only
* keytab/SSL cert stored in TPM if available
* If TPM not available, keytab/SSL cert stored on local storage (USB/disk)
* If no persistent storage available, just attach the smallest thumb drive
   that you have to the box. (i.e. *make* persistent storage available)

>> 2. Installation
>>
>> We have to account for two methods of distributing Ovirt: pre-installation 
>> by OEMs and install by IT admins.  Pre-install by OEM implies that the 
>> platform has flash or a reserved partition on the primary hard disk. 
>> Install by IT can be done by setting up PXE boot, adding an external 
>> bootable USB disk, booting to CD or installing to onboard flash/disk.
>>
>> The livecd/liveusb disk should allow either booting and running directly or 
>> installation onto platform flash/disk.
> 
> That's a nice addition, yeah. Although again I'm in favor of the
> running image itself remaining stateless.

The image is still stateless in this scenario.  It's just transferring the 
boot image from removable media to onboard media.  Once it is transferred, 
you remove the usb/cd media.

>> 3. Persistent Configuration Storage
>>
>> Since the host is non-persistent and may or may not have access to 
>> persistent storage, this could become tricky.  It's further complicated by 
>> the need to support DHCP and static addressing of the host.
>>
>> The following things need to be stored for the host:
>> a. Kerberos keytab or SSL Certificate
>> b. IP configuration (if using static configuration)
>> c. IP address of FreeIPA server and Ovirt Management Server (if using
>>    static configuration)
> 
>> Here's where things could be much simpler if DHCP were required.  Without 
>> static addressing, b is not necessary and c can be transmitted as custom 
>> DHCP fields (iirc).  Then the only thing that needs to be cached on the 
>> host on persistent storage is the auth credentials.  (NOTE: Does 
>> Microsoft's DHCP server allow custom DHCP fields???)
> 
> Actually we've talked about moving away from overloading DHCP for this
> stuff and instead handling it with mdns (let zeroconf be
> zeroconf). 

Echoing conversations on the IRC channel...

zeroconf/avahi is not reliable, so even on a small scale it is probably 
not a good idea.  dhcp fields work fine, but may not play well with MS 
DHCP servers.

So in the short term, we support sending out critical config info (ip 
addresses of certain servers) via dhcp fields.

For environments where we can't add custom dhcp fields, perhaps we should 
set up well known host aliases.  i.e. if the dhcp fields are not present 
we assume that the freeipa server is located at freeipa.domain.com (where 
domain.com is the domain specified in dhcp) and the ovirt mgmt server is 
ovirt-mgmt.domain.com.

This would mean that DNS records would need to be set up for these two 
servers, but we're already asking people to muck with DNS so....

> There is another interesting possibility we might want to look at in the
> medium term. We are actively working on ways to use amqp as part of
> our infrastructure. If a machine has an identity via a keytab and an
> IP address via DHCP, I imagine it's possible to retrieve the remaining
> necessary config information by sending a message to a known alias?
> This relieves the oVirt host of having to know the precise address of
> the management server. Something to think about at least.

Interesting idea.

>> 4. Initial Configuration
>>
>> When the host comes up the first time it needs to know the IP address and 
>> password of the Ovirt Mgmt server so it can get its auth credentials set 
>> up.  Here's how a typical setup might look:
>>
>> a. Cable up the new hardware and get MAC addrs for each iface.  Need
>>    to record which MAC is for:
>>    * management network (NOTE: should be only iface that PXE boots
>>      if PXE is being used)
>>    * storage network
>>    * normal networks
>>    (these all don't have to be separate, there could be just a single
>>    interface on the box used for management, storage and normal traffic)
>> b. Set up DNS/DHCP servers.  If using static addressing, skip DHCP setup
>>    (This is a manual step that IT admin would do, i.e. we're not trying
>>    to automate this process)
>> c. Boot Ovirt Host for first time.  Kernel cmdline takes options like:
>>    * ovirt_net=static (to indicate that static ip config should be
>>      prompted for during boot)
>>    * ovirt_auth_pw=<password> (password to use when connecting to the
>>      Ovirt Mgmt server to register the host and retrieve auth information)
>>    * ovirt_init (this is specified to let the host know that initial
>>      config needs to happen.  Otherwise normal boot occurs)
> 
> Getting back to the stateless idea, I think the host image should
> always boot exactly the same way and be smart enough to know whether
> it needs config information or not and, if so, ask for it? Having said
> that the "password" idea is reasonable except that you would always
> need to be there to enter the password for a reboot...

The password is only used to register the host on firstboot and to 
securely retrieve the keytab/SSL Cert from the management server.

So, if as above we eliminate the need for static IP config, we can reduce 
the cmdline options to:

* ovirt_auth=<password>
* ovirt_auth=floppy:/keytab
* ovirt_auth=hd:sdb1:/keytab

So either you specify a password to contact the ovirt host for 
registration and keytab retrieval or you specify the location of the 
keytab on a floppy/usb thumbdrive (syntax similar to kickstart cmdline syntax)

>> f. The host is checked for persistent storage and TPM devices.  If TPM
>>    is found it is initialized so that auth info can be stored there.
>>    * If either USB or hard disk is found it is checked for a partition
>>      with a label called "OVIRT".  If that exists it is used for
>>      persistent storage.  If it does not, an "OVIRT" partition is created.
>>      on available unpartitioned space. (how big should it be?)
>>    * USB/disk is also checked for swap partition.  If none is found, a
>>      swap partition is created using unpartitioned space.
> 
> Hadn't thought about using TPM for the keytab, that's a neat idea. I'm
> a bit leery of using any local HDD on the machine for it though.

Why?  Are you ok with using USB storage but not ok with using HDD storage?

Perry