[Ovirt-devel] [PATCH] Add support for vendor hooks during ovirt-early start()

[Ryan Harper]

* Perry Myers <pmyers at redhat.com> [2009-11-16 15:48]:
> On 11/16/2009 04:40 PM, Ryan Harper wrote:
> > * Perry Myers <pmyers at redhat.com> [2009-11-16 14:56]:
> >> On 11/16/2009 02:21 PM, Ryan Harper wrote:
> >>> Add a kernel parameter, vendor= which takes a path to a script
> >>> embedded in the image.  If this script is executable, we will then
> >>> source it during ovirt-early start() after command line processing
> >>> and before mounting of /config.  We also call a post hook at the end of
> >>> ovirt-early start().
> >>>
> >>> Also include a sample vendor script.
> >>
> >> If you're using edit-livecd to embed a vendor specific script in the oVirt
> >> ISO image, why wouldn't you just change the init process to call that
> >> script instead of using a kernel cmdline parameter like this?
> >>
> >> i.e. edit-livecd to add a new init script called vendor-foo to
> >> /etc/init.d, then chkconfig that script on.  Now during startup the script
> >> will automatically execute.
> >>
> >> I suppose maybe some information about the use case that is driving this
> >> request might help to explain.
> > 
> > Sure, one of the use cases we need is using an iscsi target as "local"
> > disk.  This requires bringing up networking and iscsi prior to
> > ovirt-early trying to mount the /config partition.  Previously, had just
> > injected code around the area where the pre hook is now.  Recently moved
> > that to it's own function so we could conditionally call it.. then
> > looking at how to merge any of that upstream and it made more sense to
> > keep the very vendory specific changes out of ovirt-early altogether.  I
> > hadn't thought to do a completely separate file since some of the logic
> > triggers on other boot parameters that ovirt-early processes (like
> > firstboot, and storage_init).
> 
> jboggs was working on providing iscsi as local disk and the plan was to
> integrate that as fully supported functionality for the base oVirt ISO
> image.  So I'd prefer in this particular case that we work to get it
> integrated into the base oVirt ISO image rather than put it in as a vendor
> specific script.

I can share what we've had to do to get it working so far; not the
prettiest bit of code.  

Networking must be brough up prior to iscsi connection and one can't
access /config.  Ideally, we could pass a simple CONFIG_IP_PNP parameter
to the kernel to get networking autoconfigured but neither RHEL nor
Fedora enable IP_PNP.  Passing the config via command is doable, but
makes having a common boot image/config troublesome since each server
has to have a unique config file.  Additionally, we wanted to configure
a bonded bridge prior to establishing the iscsi connection since once
you mount /config over iscsi one cannot restart networking without
dropping your filesystem into RO mode.  The per-server config issue
ended up being resolved by embedding additional config (iscsi server,
initiatorname, and nfs server path) via DHCP vendor options.  
Here is what we're doing to bring up networking and connect to an iscsi 
target prior to mounting /config; this code runs where vendor_pre_hook()
would.


# don't let networking autostart later
chkconfig --del network

# generate custom ifcfg network files that
#   - build a bonded interface over 10g nics
#   - create a bridge with the bond0 as port
configure_network_files

# disable any current networking
service network restart

# look for custom DHCP paramters if they exist
# and extract iscsi server IP and initator name
echo "InitiatorName=${iscsi_initiator}" > /etc/iscsi/initiatorname.iscsi

# disable iscsi autostart
chkconfig --del iscsi
chkconfig --del iscsd

# bring up iscsi
service iscsi start

# discover iscsi targets on iscsi server
TARGET=$(iscsiadm -m discovery -t sendtargets -p ${iscsi_server})

# restart iscsi server
server iscsi restart

# wait for devices to populate /dev/disk/by-id
l=0
count=`find /dev/disk/by-id -mindeth 1 2>/dev/null | wc -l`
while [ $l -lt 5 ]; do
    curr=`find /dev/disk/by-id -mindeth 1 2>/dev/null | wc -l`
    log "by-id now has $curr entries"
    [ $curr -gt $count ] && break
    l=$(($l + 1))
    sleep 1;
done

# if not firstboot, expect LVs 
if [ "$firstboot" != "1" ]; then
    # it's not the firstboot
    log "Scanning for lvm config"
    l=0
    while [ $l -lt 3 ]; do
        echo "lvm scan count=$l" >> /tmp/lvm.log
        pvscan >> /tmp/lvm.log
        sleep 1
        vgscan >> /tmp/lvm.log
        sleep 1
        lvscan >> /tmp/lvm.log
        sleep 1
        vgchange -ay >> /tmp/lvm.log
        [ -e /dev/HostVG/Config ] && break;
        l=$(($l + 1))
    done
    if [ ! -e /dev/HostVG/Config ] ; then
        log "Rescue mode requested, starting emergency shell"
        stop_log
        bash < /dev/console
        start_log
    fi
fi

At this point, we've made an iscsi connection and repopulated LVM and we can mount /config.


> 
> apevec, since jboggs is out of the office for a bit can you follow up here?
> 
> > I'll have to look at seeing if we could move that into it's own init
> > script; though there might be some duplication of code (command parsing,
> > and such).  With the vendor approach, it executes in the same context as
> > ovirt-early so one can potentially override function implementations and
> > exert control over the flow in the remainder of ovirt-early; that
> > wouldn't be doable in a separate init script.  
> > 
> > One example that we rely upon that can't be done from separate script is
> > appending boot parameters to the params list to be saved.
> 
> ... But regardless, it is probably worth creating an easy to use interface
> for vendors to add their own init scripts in without needing to duplicate
> code, and trampling on ovirt-early.  Perhaps an
> /etc/insert-your-vendor-script-here.d directory that ovirt-early and
> ovirt-post reads and executes functions from?  That would provide an easy
> way to drop in custom functionality and would also not require a new
> kernel cmdline param.  Thoughts?

That does seem cleaner; though I do still like having a way to choose
which set of vendor scripts to run via kernel parameter.  We'd like to
have a single iso image that we could support different types of
startups.


-- 
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
ryanh at us.ibm.com