[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Ovirt-devel] [PATCH] oVirt Node local installation



Perry N. Myers wrote:
 /config - local oVirt configuration (ktab, local admin password...)
 /boot - boot loader, kernel and initramfs
 /LiveOS - oVirt Node compressed livecd image

Is the initramfs repeated twice then? Once inside of the LiveCD and once in /boot? Or am I just reading this wrong...

There's just one, livecd-tools created, initrd0.img, LiveOS contains only squashfs.img containing ext3fs.img containing real root fs.

  ovirt_init=usb|scsi[:serial#]
So if you leave off serial#, how does it choose the disk? Does it just take the first device (i.e. /dev/sda) and start using it?

Yes, it will take the first device, skipping current /dev/live and format it.
As per IRC discussion, ovirt_init syntax is changed to: ovirt_init[=usb|scsi[:serial#]]
just 'ovirt_init' w/o value will initialize /config on /dev/live if writable

[1] one-liner to list serial#s for all disks in the system:
for d in /dev/sd?; do eval $(udevinfo --query env --name $d); echo $d $ID_SERIAL;done

Problem with this is it assumes someone has booted linux on the box before and has access to the above command. I think for the most part people will not have the serial #s to work with. It's fine to leave this functionality in, I just wonder how much it would be used.

In theory, external USB flash could be plugged into admin's workstation running Linux, and if it's internal flash device, OEM might have a way to predict the serial#
But you're right, I don't have really useful use-case for serial#. I guess we can still leave it there and document only in troubleshooting or advanced doc chapter.

Other question is... Should we have a sane default in the init scripts? i.e. even if you don't pass in ovirt_init the init script does the following:

1. Looks to see if there is an existing OVIRT partition and if so mounts
   it
2. If one does not exist looks for the first hard disk with free space on
   it and creates the OVIRT partition there.
3. If no hard drive space is available then it looks for the first flash
   disk with free space on it and creates the OVIRT partition there.
These partitions are for config only, unless the ovirt_local_boot was passed on the cmdline, in which case the local installation is done. This way the admin is not forced to always use the ovirt_init parameter on every machine in the datacenter. The node will do the right thing if the OVIRT partition is not already there.

I don't think we should make any local changes w/o explicit admin action (i.e. ovirt_* parameter now, local console config utility later) - for mass deployment admin will use PXE and cobbler, allowing him to specify boot parameters per machine and for SMB use-case we could increase usability by providing boot menu option "Install to local storage"

Also, steps 2 and 3 can be used to set up local swap if it doesn't already exist.

Local swap will be allocated from a logical volume created from remaining local disk space.
This is TBD, I'll address it as a separate patch.

Also, if ovirt_local_boot is passed on the command line, the OVIRT partition should never be placed on the same flash device as the one that was booted since that device is transient.

yes, device pointed by /dev/live symlink is skipped

+ls -l /dev/disk
+        mkdir -p /dev/disk/by-label
+ls -l /dev/disk/by-label

Are the two ls lines above missing indentation?

debugging leftover, I'll remove that

+      cat > $ovirt/boot/grub/grub.conf <<EOF
+default=0
+timeout=2
+hiddenmenu
+title oVirt Managed Node
+    root (hd0,$part_num)
+    kernel /boot/vmlinuz0 ro root=LABEL=OVIRT roottypefs=ext2 liveimg
+    initrd /boot/initrd0.img
+EOF

Ian will need to add to this patch later for some serial console stuff probably.

fixed by passing console= parameters from the current /proc/cmdline and adding them to grub entry

attached is the revised patch I'm about to push
>From 567afde6fa3abddbc024dc4889325c5a352c0934 Mon Sep 17 00:00:00 2001
From: Alan Pevec <apevec redhat com>
Date: Fri, 1 Aug 2008 15:55:44 +0200
Subject: [PATCH] oVirt Node local installation

oVirt partition can be initialized on a local flash or disk drive.
oVirt partition is a local primary ext2 partition, labeled OVIRT,
which contains:
 /config - local oVirt configuration (ktab, local admin password...)
 /boot - boot loader, kernel and initramfs
 /LiveOS - oVirt Node compressed livecd image

oVirt partition is initialized using kernel boot parameters:
  ovirt_init[=usb|scsi[:serial#]]
local installation target disk
usb|scsi - select disk type, as reported by udev ID_BUS
serial# - select exact disk using serial number, as reported by
          udev ID_SERIAL [1]
e.g. ovirt_init=usb:Generic_STORAGE_DEVICE_0000145418-0:0

  ovirt_local_boot
install/update oVirt Node image on the local installation target disk
without this parameter oVirt partition is used for storing
oVirt configuration only

[1] one-liner to list serial#s for all disks in the system:
for d in /dev/sd?; do eval $(udevinfo --query env --name $d); echo $d $ID_SERIAL;done

Signed-off-by: Alan Pevec <apevec redhat com>
---
 ovirt-host-creator/common-pkgs.ks              |    1 -
 ovirt-managed-node/src/scripts/ovirt           |   66 +++++--
 ovirt-managed-node/src/scripts/ovirt-awake     |   10 +-
 ovirt-managed-node/src/scripts/ovirt-early     |  278 +++++++++++++++++++++++-
 ovirt-managed-node/src/scripts/ovirt-functions |    3 +
 ovirt-managed-node/src/scripts/ovirt-post      |    2 +-
 6 files changed, 331 insertions(+), 29 deletions(-)

diff --git a/ovirt-host-creator/common-pkgs.ks b/ovirt-host-creator/common-pkgs.ks
index 2d1ad6e..bc08358 100644
--- a/ovirt-host-creator/common-pkgs.ks
+++ b/ovirt-host-creator/common-pkgs.ks
@@ -48,7 +48,6 @@ ovirt-managed-node
 -rhpl
 -kbd
 -usermode
--grub
 -fedora-logos
 -kpartx
 -dmraid
diff --git a/ovirt-managed-node/src/scripts/ovirt b/ovirt-managed-node/src/scripts/ovirt
index 8f15dff..ba1f20c 100755
--- a/ovirt-managed-node/src/scripts/ovirt
+++ b/ovirt-managed-node/src/scripts/ovirt
@@ -15,36 +15,74 @@ start() {
     # and it needs to be at the front of the forward chain
     iptables -I FORWARD -m physdev --physdev-is-bridged -j ACCEPT
 
+    krb5_conf=/etc/krb5.conf
+    krb5_tab=/etc/libvirt/krb5.tab
+    # retrieve config from local oVirt partition if available
+    #   krb5.conf krb5.tab
+    #   TODO local admin password, ssh server key - what else?
+    ovirt=$(mktemp -d)
+    cfg=$ovirt/config
+    if [ -e /dev/disk/by-label/$OVIRT_LABEL ]; then
+      mount -r /dev/disk/by-label/$OVIRT_LABEL $ovirt
+    else
+      mount -r /dev/live $ovirt
+    fi
+    if [ -e $cfg/krb5.conf ]; then
+      cp -a $cfg/krb5.conf $krb5_conf
+    fi
+    if [ -e $cfg/krb5.tab ]; then
+      cp -a $cfg/krb5.tab $krb5_tab
+    fi
+    if [ -s $krb5_tab ]; then
+      krb5_tab=
+    fi
+
     find_srv ipa tcp
     if [ -n "$SRV_HOST" -a -n "$SRV_PORT" ]; then
-        krb5_conf=/etc/krb5.conf
-        if [ ! -s $krb5_conf ]; then
-            rm -f $krb5_conf
-            # FIXME this is IPA specific
-            wget -q \
-                http://$SRV_HOST:$SRV_PORT/ipa/config/krb5.ini -O $krb5_conf \
-                || (echo "Failed to get $krb5_conf" && return 1)
+        # FIXME this is IPA specific
+        wget -q \
+            http://$SRV_HOST:$SRV_PORT/ipa/config/krb5.ini -O $krb5_conf.tmp
+        if [ $? -ne 0 ]; then
+            echo "Failed to get $krb5_conf"; return 1
+        fi
+        mv $krb5_conf.tmp $krb5_conf
+        # store config in oVirt partition
+        if [ -e $cfg ]; then
+            mount -o remount,rw $ovirt
+            cp -a $krb5_conf $cfg/krb5.conf
         fi
     else
-        echo "skipping Kerberos configuration, IPA service not available"
+        echo "skipping Kerberos configuration"
     fi
 
     find_srv identify tcp
     if [ -n "$SRV_HOST" -a -n "$SRV_PORT" ]; then
-        krb5_tab=/etc/libvirt/krb5.tab
-        ovirt-awake start $krb5_tab $SRV_HOST $SRV_PORT
+        ovirt-awake start $SRV_HOST $SRV_PORT $krb5_tab
+        if [ $? -ne 0 ]; then
+            echo "ovirt-awake failed"; return 1
+        fi
+        # store config in oVirt partition
+        if [ -n "$krb_tab" -a -e $cfg ]; then
+            mount -o remount,rw $ovirt
+            cp -a $krb5_tab $cfg/krb5.tab
+        fi
     else
-        echo "skipping ovirt-awake, oVirt registration service not available"
+        echo "skipping ovirt-awake, oVirt identify service not available"
     fi
 
+    # cleanup
+    umount $ovirt && rmdir $ovirt
+
     find_srv collectd tcp
     if [ -n "$SRV_HOST" -a -n "$SRV_PORT" ]; then
         collectd_conf=/etc/collectd.conf
         if [ -f $collectd_conf.in ]; then
             sed -e "s/@COLLECTD_SERVER@/$SRV_HOST/" \
                 -e "s/@COLLECTD_PORT@/$SRV_PORT/" $collectd_conf.in \
-                > $collectd_conf \
-                || (echo "Failed to write $collectd_conf" && return 1)
+                > $collectd_conf
+            if [ $? -ne 0 ]; then
+                echo "Failed to write $collectd_conf"; return 1
+            fi
         fi
     else
         echo "skipping collectd configuration, collectd service not available"
@@ -53,7 +91,7 @@ start() {
 
 case "$1" in
     start)
-        echo -n $"Starting ovirt: "
+        printf "Starting ovirt: "
 
         {
             start
diff --git a/ovirt-managed-node/src/scripts/ovirt-awake b/ovirt-managed-node/src/scripts/ovirt-awake
index 38d405e..fc03c9a 100755
--- a/ovirt-managed-node/src/scripts/ovirt-awake
+++ b/ovirt-managed-node/src/scripts/ovirt-awake
@@ -63,10 +63,10 @@ start () {
 
             KEYTAB=`echo $REPLY | awk '{ print $2 }'`
 
-            if [ -n $KEYTAB ]; then
+            if [ -n "$KEYTAB" -a -n "$KEYTAB_FILE" ]; then
                 echo "Retrieving keytab: '$KEYTAB'"
 
-                wget -q $KEYTAB --output-document=$KEYTAB_FILE
+                wget -q "$KEYTAB" --output-document="$KEYTAB_FILE"
             else
                 echo "No keytab to retrieve"
             fi
@@ -84,9 +84,9 @@ start () {
 
 case "$1" in
     start)
-        KEYTAB_FILE=$2
-        SRV_HOST=$3
-        SRV_PORT=$4
+        SRV_HOST=$2
+        SRV_PORT=$3
+        KEYTAB_FILE=$4
         start
         RETVAL=$?
     ;;
diff --git a/ovirt-managed-node/src/scripts/ovirt-early b/ovirt-managed-node/src/scripts/ovirt-early
index 44e8b8e..b709632 100755
--- a/ovirt-managed-node/src/scripts/ovirt-early
+++ b/ovirt-managed-node/src/scripts/ovirt-early
@@ -10,6 +10,9 @@
 . /etc/init.d/functions
 . /etc/init.d/ovirt-functions
 
+# size of the oVirt partition in megabytes
+OVIRT_SIZE=64
+
 configure_from_network() {
     DEVICE=$1
     if [ -n "$DEVICE" ]; then
@@ -56,21 +59,280 @@ configure_from_network() {
     printf "default config applied."
 }
 
+# find_disk $bus $serial $live_disk
+find_disk() {
+    local bus=$1
+    local serial=$2
+    local live=$3
+    for d in /dev/sd?; do
+      eval $(udevinfo --query env --name $d)
+      if [ "$ID_BUS" = "$bus" ]; then
+        if [ -z "$serial" -o "$ID_SERIAL" = "$serial" ]; then
+          if [ -n "$live" -a "$d" = "$live" ]; then
+            # cannot install LiveOS over itself
+            continue
+          else
+            echo $d
+            return 0
+          fi
+        fi
+      fi
+    done
+    return 1
+}
+
+# local_install $local_os $target
+#  local_os - 1=install LiveOS and boot loader
+#             0=initialize oVirt partition only
+#  target - target disk to hold the oVirt partition
+#           =usb|scsi[:serial#]
+#  bootparams - extra boot parameters like console=
+#
+# oVirt partition is a local primary ext2 partition, labeled OVIRT
+# content:
+#  /config - local oVirt configuration (ktab, local admin password)
+#  /boot - boot loader, kernel and initramfs
+#  /LiveOS - oVirt Node compressed livecd image
+
+local_install() {
+    local local_os=$1
+    local target=$2
+    local bootparams=$3
+    local disk
+    local part
+    local live_part
+    local live_disk
+    local live_dev=/dev/live
+    if [ ! -e $live_dev ]; then
+      # PXE boot
+      live_dev=/dev/loop0
+      live_part=$live_dev
+      live_disk=
+    else
+      live_part=$(readlink -e $live_dev)
+      live_disk=${live_disk%[1-9]}
+    fi
+    local ovirt_part=$(readlink -e /dev/disk/by-label/$OVIRT_LABEL)
+    local ovirt_disk=${ovirt_part%[1-9]}
+    if [ "$ovirt_disk" = "$ovirt_part" ]; then
+      ovirt_disk=
+    fi
+    if [ -z "$target" ]; then
+        if [ -z "$ovirt_disk" ]; then
+          return 1
+        fi
+        # ovirt_init not specified, use pre-labeled oVirt partition
+        mode=update
+        disk=$ovirt_disk
+        part=$ovirt_part
+    else
+      case "$target" in
+          =)
+          # empty ovirt_init, use current live image device
+          mode=update
+          disk=$live_disk
+          part=$live_part
+          ;;
+          =scsi*)
+          bus=scsi
+          serial=${target#=scsi:}
+          mode=install
+          ;;
+          =usb*)
+          bus=usb
+          serial=${target#=usb:}
+          mode=install
+          ;;
+          *)
+          return 1
+          ;;
+      esac
+      if [ $mode = "install" ]; then
+        if [ "$serial" = "=$bus" ]; then
+          serial=
+        fi
+        disk=$(find_disk $bus $serial $live_disk)
+        rc=$?
+        if [ $rc -ne 0 ]; then
+          echo "target disk '$target' not available"
+          return 1
+        fi
+        if [ -n "$ovirt_disk" ]; then
+          if [ "$disk" = "$ovirt_disk" ]; then
+            # target disk contains oVirt partition, select it for update
+            # TODO force reinstall option
+            mode=update
+            part=$ovirt_part
+          else
+            # remove label from oVirt partition, there can be only one
+            e2label $ovirt_part ""
+          fi
+        fi
+      fi
+      if [ "$mode" = "install" ]; then
+        printf "installing $disk..." | tee /dev/console
+        dd if=/dev/zero bs=4096 count=1 of=$disk \
+        && parted -s $disk mklabel msdos \
+        && parted -s $disk mkpart primary ext2 0.0 $OVIRT_SIZE \
+        && partprobe -s $disk
+        if [ $? -ne 0 ]; then
+          echo "$disk partition creation failed"; return 1
+        fi
+        part=${disk}1
+        udevsettle
+        mkfs.ext2 -L $OVIRT_LABEL $part \
+        && tune2fs -c0 -i0 $part
+        if [ $? -ne 0 ]; then
+          echo "$disk ext2 creation failed"; return 1
+        fi
+        # update by-label link manually, mkfs won't trigger udev
+        mkdir -p /dev/disk/by-label
+        ln -sf $part /dev/disk/by-label/$OVIRT_LABEL
+      fi
+    fi
+
+    if [ "$mode" = "update" ]; then
+      printf "updating $disk..." | tee /dev/console
+    fi
+    ovirt=$(mktemp -d)
+    if [ "$part" = "$live_part" ]; then
+      # ovirt_init w/o specified target
+      # setup /config on live disk, if writeable
+      # TODO mlabel/e2label (check fs2 type or just blindly try?)
+      mount -r $part $ovirt && mount -o remount,rw $ovirt \
+      && mkdir -p $ovirt/config
+      umount $ovirt && rmdir $ovirt
+      return 0
+    fi
+    live=$(mktemp -d)
+    mount -r $live_dev $live
+    if [ $? -ne 0 ]; then
+      echo "source image mount failed"; return 1
+    fi
+    mount $part $ovirt
+    if [ $? -ne 0 ]; then
+      echo "target mount failed"; return 1
+    fi
+    mkdir -p $ovirt/config
+    # update local config using the one embedded in livecd image
+    # TODO admin tool for adding /config into livecd image
+    if [ -e $live/config/krb5.conf ]; then
+      cp -a $live/config/krb5.conf $ovirt/config \
+      || echo "krb5.conf copy failed"
+    fi
+    if [ -e $live/config/krb5.tab ]; then
+      cp -a $live/config/krb5.tab $ovirt/config \
+      || echo "krb5.tab copy failed"
+    fi
+
+    if [ $local_os = 0 ]; then
+      # config update only, cleanup and continue booting
+      umount $ovirt && rmdir $ovirt
+      umount $live && rmdir $live
+    else
+      # install oVirt Node image for local boot
+      if [ -e "$live/syslinux" ]; then
+        syslinux=syslinux
+      elif [ -e "$live/isolinux" ]; then
+        syslinux=isolinux
+      else
+        syslinux=
+      fi
+      rm -rf $ovirt/boot
+      rm -rf $ovirt/LiveOS
+      mkdir -p $ovirt/boot/grub
+      mkdir -p $ovirt/LiveOS
+      cp -a $live/LiveOS/squashfs.img $ovirt/LiveOS \
+      && cp -a $live/$syslinux/initrd0.img $ovirt/boot \
+      && cp -a $live/$syslinux/vmlinuz0 $ovirt/boot
+      if [ $? -ne 0 ]; then
+        echo "image copy failed"; return 1
+      fi
+      part_num=$(( ${part#$disk} - 1 ))
+      cat > $ovirt/boot/grub/grub.conf << EOF
+default=0
+timeout=2
+hiddenmenu
+title oVirt Managed Node
+    root (hd0,$part_num)
+    kernel /boot/vmlinuz0 ro root=LABEL=OVIRT roottypefs=ext2 liveimg $bootparams
+    initrd /boot/initrd0.img
+EOF
+      grub-install --root-directory=$ovirt $disk >&2
+      if [ $? -ne 0 ]; then
+        echo "boot loader install failed"; return 1
+      fi
+      # remove 1.5 stages we don't need
+      find $ovirt/boot/grub -name '*_stage1_5' ! -name e2fs_stage1_5 \
+        -exec rm {} \;
+      umount $ovirt && rmdir $ovirt
+      umount $live && rmdir $live
+      # FIXME avoid reboot loops
+      # temp. workaround: sync and wait
+      sync; sync; sync
+      printf "oVirt local installation finished, press Enter to reboot." \
+        > /dev/console
+      read key
+      if [ "$key" = "debug" ]; then
+        sh > /dev/console 2>&1
+      fi
+      reboot
+    fi
+}
+
 start() {
-    # find boot interface from cmdline
-    # IPAPPEND 2 in pxelinux.cfg appends e.g. BOOTIF=01-00-16-3e-12-34-57
-    BOOTIF=
+    # oVirt boot parameters
+    #   BOOTIF=<MAC> (appended by pxelinux)
+    #   ovirt_init=usb|scsi[:serial#]
+    #   ovirt_local_boot
+
+    #   BOOTIF=<MAC> (appended by pxelinux)
+    # network boot interface is assumed to be on management network where
+    # oVirt Server is reachable
+    # IPAPPEND 2 in pxelinux.cfg appends MAC address of the booting node
+    # e.g. BOOTIF=01-00-16-3e-12-34-57
+    bootif=
+
+    #   ovirt_init=usb|scsi[:serial#]
+    # local installation target disk
+    # usb|scsi - select disk type, as reported by udev ID_BUS
+    # serial# - select exact disk using serial number, as reported by
+    #           udev ID_SERIAL
+    # e.g. ovirt_init=usb:Generic_STORAGE_DEVICE_0000145418-0:0
+    target=
+
+    #   ovirt_local_boot
+    # install/update oVirt Node image on the local installation target disk
+    local_os=0
+
+    # save boot parameters like console= for local disk boot menu
+    bootparams=
+
     for i in $(cat /proc/cmdline); do
         case $i in
             BOOTIF=??-??-??-??-??-??-??)
-            i=${i/#BOOTIF=??-/}
-            BOOTMAC=${i//-/:}
-            BOOTIF=$(grep -l $BOOTMAC /sys/class/net/eth*/address|rev|cut -d/ -f2|rev)
+            i=${i#BOOTIF=??-}
+            bootif=$(grep -il $(echo $i|sed 's/-/:/g') /sys/class/net/eth*/address|rev|cut -d/ -f2|rev)
+            ;;
+            ovirt_init*)
+            target=${i#ovirt_init}
+            if [ -z "$target" ]; then
+              target='='
+            fi
+            ;;
+            ovirt_local_boot*)
+            local_os=1
+            ;;
+            console=*)
+            bootparams="$bootparams $i"
             ;;
         esac
     done
 
-    configure_from_network $BOOTIF
+    set -x
+    local_install "$local_os" "$target" "$bootparams"
+    set +x
+    configure_from_network $bootif
 
     # find all of the partitions on the system
 
@@ -101,7 +363,7 @@ start() {
 
 case "$1" in
     start)
-        echo -n $"Starting ovirt-early: "
+        printf "Starting ovirt-early: "
 
         {
             start
diff --git a/ovirt-managed-node/src/scripts/ovirt-functions b/ovirt-managed-node/src/scripts/ovirt-functions
index 5b530f7..083e13d 100644
--- a/ovirt-managed-node/src/scripts/ovirt-functions
+++ b/ovirt-managed-node/src/scripts/ovirt-functions
@@ -2,6 +2,9 @@
 
 OVIRT_LOGFILE=/var/log/ovirt.log
 
+# label of the oVirt partition
+OVIRT_LABEL=OVIRT
+
 find_srv()
 {
     local dnsreply
diff --git a/ovirt-managed-node/src/scripts/ovirt-post b/ovirt-managed-node/src/scripts/ovirt-post
index ecc6ff4..1267f40 100755
--- a/ovirt-managed-node/src/scripts/ovirt-post
+++ b/ovirt-managed-node/src/scripts/ovirt-post
@@ -22,7 +22,7 @@ start() {
 
 case "$1" in
     start)
-        echo -n $"Starting ovirt-post: "
+        printf "Starting ovirt-post: "
 
         {
             start
-- 
1.5.5.1


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]