[Date Prev][Date Next] [Thread Prev][Thread Next]
Re: [Ovirt-devel] Ovirt Host Tasks
- From: "Perry N. Myers" <pmyers redhat com>
- To: ovirt-devel redhat com
- Subject: Re: [Ovirt-devel] Ovirt Host Tasks
- Date: Mon, 17 Mar 2008 15:02:20 -0400
Hugh O. Brock wrote:
On Sun, Mar 16, 2008 at 11:43:02PM -0400, Perry N. Myers wrote:
Wow that's a big mail <g>.
I have a couple of general comments based on the way we *have been*
thinking about this stuff (this of course doesn't mean we can't
First off, we have been imagining the oVirt host image as stateless
except for whatever we use to store its identity (if we even do
that). This means we simply don't support static addressing of oVirt
hosts, at all. The appealing thing about a stateless oVirt host image
to me is that it means the host is immediately and easily upgradeable;
there is 0 local state to worry about aside from the Kerberos
keytab. It also means that a reboot will immediately restore a
compromised machine FWIW.
Having said all that, if there is a compelling reason to move away
from the stateless idea I'm all ears.
I'm all for getting rid of static config support. The fundamental
question is, how many people would require static IP config (as opposed to
static DHCP reservations)? If that is a significant number of potential
users, it makes sense to support it.
We can always take the approach of: implement stateless/DHCP support only
and then if we get feedback from enough folks that static addressing is
important enough we can try to add it in later.
Unless I hear major objections, this is how I'll proceed.
Further comments in-line below...
1. Image Creation:
Right now the Ovirt Host is created using scripts in the Ovirt host
repository that utilize livecd creator to make ISO, USB and PXE formats.
I think it makes sense to start with the tools that Chris L. has already
been working with and migrate new functionality in, rather than starting
with other tools.
a. Integrate whitelist functionality into the image creation
process to reduce image size.
b. Determine the minimal set of files that the host image needs to
construct a baseline whitelist.
c. Determine minimal set of kernel modules to support wide platform base
(i.e. keep disk, net modules. remove things like sound, isdn, etc...)
d. Construct a repeatable build process so that images can be built
daily for testing.
e. Construct a utility that detects the minimal set of kernel modules
for a given platform and generates a config file that can be fed to
the build system for generating hardware platform specific images.
f. Construct a method for taking the generic host image and removing all
kernel modules not applicable for a specific platform.
g. Construct a method for taking an image and adding site specific
configuration information to it (IP addresses of Ovirt Mgmt Server and
FreeIPA Server). This is only necessary to support hosts that require
static IP configuration -and- do not have persistent storage to store
configuration information. (If we decide to support static IP config
we could make presence of persistent storage a requirement...)
You could add "Kerberos keytab" to "site specific configuration
information" and the host image would be completely
self-contained. I can imagine a library of unique images under
I don't like the idea of having a separate PXE image for each individual
host unless -absolutely- necessary. So we shouldn't encourage this.
So here are some assumptions we would should make:
* DHCP only
* keytab/SSL cert stored in TPM if available
* If TPM not available, keytab/SSL cert stored on local storage (USB/disk)
* If no persistent storage available, just attach the smallest thumb drive
that you have to the box. (i.e. *make* persistent storage available)
We have to account for two methods of distributing Ovirt: pre-installation
by OEMs and install by IT admins. Pre-install by OEM implies that the
platform has flash or a reserved partition on the primary hard disk.
Install by IT can be done by setting up PXE boot, adding an external
bootable USB disk, booting to CD or installing to onboard flash/disk.
The livecd/liveusb disk should allow either booting and running directly or
installation onto platform flash/disk.
That's a nice addition, yeah. Although again I'm in favor of the
running image itself remaining stateless.
The image is still stateless in this scenario. It's just transferring the
boot image from removable media to onboard media. Once it is transferred,
you remove the usb/cd media.
3. Persistent Configuration Storage
Since the host is non-persistent and may or may not have access to
persistent storage, this could become tricky. It's further complicated by
the need to support DHCP and static addressing of the host.
The following things need to be stored for the host:
a. Kerberos keytab or SSL Certificate
b. IP configuration (if using static configuration)
c. IP address of FreeIPA server and Ovirt Management Server (if using
Here's where things could be much simpler if DHCP were required. Without
static addressing, b is not necessary and c can be transmitted as custom
DHCP fields (iirc). Then the only thing that needs to be cached on the
host on persistent storage is the auth credentials. (NOTE: Does
Microsoft's DHCP server allow custom DHCP fields???)
Actually we've talked about moving away from overloading DHCP for this
stuff and instead handling it with mdns (let zeroconf be
Echoing conversations on the IRC channel...
zeroconf/avahi is not reliable, so even on a small scale it is probably
not a good idea. dhcp fields work fine, but may not play well with MS
So in the short term, we support sending out critical config info (ip
addresses of certain servers) via dhcp fields.
For environments where we can't add custom dhcp fields, perhaps we should
set up well known host aliases. i.e. if the dhcp fields are not present
we assume that the freeipa server is located at freeipa.domain.com (where
domain.com is the domain specified in dhcp) and the ovirt mgmt server is
This would mean that DNS records would need to be set up for these two
servers, but we're already asking people to muck with DNS so....
There is another interesting possibility we might want to look at in the
medium term. We are actively working on ways to use amqp as part of
our infrastructure. If a machine has an identity via a keytab and an
IP address via DHCP, I imagine it's possible to retrieve the remaining
necessary config information by sending a message to a known alias?
This relieves the oVirt host of having to know the precise address of
the management server. Something to think about at least.
4. Initial Configuration
When the host comes up the first time it needs to know the IP address and
password of the Ovirt Mgmt server so it can get its auth credentials set
up. Here's how a typical setup might look:
a. Cable up the new hardware and get MAC addrs for each iface. Need
to record which MAC is for:
* management network (NOTE: should be only iface that PXE boots
if PXE is being used)
* storage network
* normal networks
(these all don't have to be separate, there could be just a single
interface on the box used for management, storage and normal traffic)
b. Set up DNS/DHCP servers. If using static addressing, skip DHCP setup
(This is a manual step that IT admin would do, i.e. we're not trying
to automate this process)
c. Boot Ovirt Host for first time. Kernel cmdline takes options like:
* ovirt_net=static (to indicate that static ip config should be
prompted for during boot)
* ovirt_auth_pw=<password> (password to use when connecting to the
Ovirt Mgmt server to register the host and retrieve auth information)
* ovirt_init (this is specified to let the host know that initial
config needs to happen. Otherwise normal boot occurs)
Getting back to the stateless idea, I think the host image should
always boot exactly the same way and be smart enough to know whether
it needs config information or not and, if so, ask for it? Having said
that the "password" idea is reasonable except that you would always
need to be there to enter the password for a reboot...
The password is only used to register the host on firstboot and to
securely retrieve the keytab/SSL Cert from the management server.
So, if as above we eliminate the need for static IP config, we can reduce
the cmdline options to:
So either you specify a password to contact the ovirt host for
registration and keytab retrieval or you specify the location of the
keytab on a floppy/usb thumbdrive (syntax similar to kickstart cmdline syntax)
f. The host is checked for persistent storage and TPM devices. If TPM
is found it is initialized so that auth info can be stored there.
* If either USB or hard disk is found it is checked for a partition
with a label called "OVIRT". If that exists it is used for
persistent storage. If it does not, an "OVIRT" partition is created.
on available unpartitioned space. (how big should it be?)
* USB/disk is also checked for swap partition. If none is found, a
swap partition is created using unpartitioned space.
Hadn't thought about using TPM for the keytab, that's a neat idea. I'm
a bit leery of using any local HDD on the machine for it though.
Why? Are you ok with using USB storage but not ok with using HDD storage?
[Date Prev][Date Next] [Thread Prev][Thread Next]