[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Ovirt-devel] [PATCH] oVirt Node local installation



Alan Pevec wrote:
oVirt partition can be initialized on a local flash or disk drive.
oVirt partition is a local primary ext2 partition, labeled OVIRT,
which contains:
 /config - local oVirt configuration (ktab, local admin password...)
 /boot - boot loader, kernel and initramfs
 /LiveOS - oVirt Node compressed livecd image

Is the initramfs repeated twice then? Once inside of the LiveCD and once in /boot? Or am I just reading this wrong...

oVirt partition is initialized using kernel boot parameters:
  ovirt_init=usb|scsi[:serial#]
local installation target disk
usb|scsi - select disk type, as reported by udev ID_BUS
serial# - select exact disk using serial number, as reported by
          udev ID_SERIAL [1]
e.g. ovirt_init=usb:Generic_STORAGE_DEVICE_0000145418-0:0

So if you leave off serial#, how does it choose the disk? Does it just take the first device (i.e. /dev/sda) and start using it?

  ovirt_local_boot
install/update oVirt Node image on the local installation target disk
without this parameter oVirt partition is used for storing
oVirt configuration only

[1] one-liner to list serial#s for all disks in the system:
for d in /dev/sd?; do eval $(udevinfo --query env --name $d); echo $d $ID_SERIAL;done

Problem with this is it assumes someone has booted linux on the box before and has access to the above command. I think for the most part people will not have the serial #s to work with. It's fine to leave this functionality in, I just wonder how much it would be used.

Other question is... Should we have a sane default in the init scripts? i.e. even if you don't pass in ovirt_init the init script does the following:

1. Looks to see if there is an existing OVIRT partition and if so mounts
   it
2. If one does not exist looks for the first hard disk with free space on
   it and creates the OVIRT partition there.
3. If no hard drive space is available then it looks for the first flash
   disk with free space on it and creates the OVIRT partition there.

These partitions are for config only, unless the ovirt_local_boot was passed on the cmdline, in which case the local installation is done.

This way the admin is not forced to always use the ovirt_init parameter on every machine in the datacenter. The node will do the right thing if the OVIRT partition is not already there.

Also, steps 2 and 3 can be used to set up local swap if it doesn't already exist.

Also, if ovirt_local_boot is passed on the command line, the OVIRT partition should never be placed on the same flash device as the one that was booted since that device is transient.

Minor comments below...

Signed-off-by: Alan Pevec <apevec redhat com>
---
 ovirt-host-creator/common-pkgs.ks              |    1 -
 ovirt-managed-node/src/scripts/ovirt           |   64 +++++--
 ovirt-managed-node/src/scripts/ovirt-awake     |   10 +-
 ovirt-managed-node/src/scripts/ovirt-early     |  248 +++++++++++++++++++++++-
 ovirt-managed-node/src/scripts/ovirt-functions |    3 +
 ovirt-managed-node/src/scripts/ovirt-post      |    2 +-
 6 files changed, 298 insertions(+), 30 deletions(-)

diff --git a/ovirt-host-creator/common-pkgs.ks b/ovirt-host-creator/common-pkgs.ks
index 2d1ad6e..bc08358 100644
--- a/ovirt-host-creator/common-pkgs.ks
+++ b/ovirt-host-creator/common-pkgs.ks
@@ -48,7 +48,6 @@ ovirt-managed-node
 -rhpl
 -kbd
 -usermode
--grub
 -fedora-logos
 -kpartx
 -dmraid
diff --git a/ovirt-managed-node/src/scripts/ovirt b/ovirt-managed-node/src/scripts/ovirt
index 92ed330..b6dcbc6 100755
--- a/ovirt-managed-node/src/scripts/ovirt
+++ b/ovirt-managed-node/src/scripts/ovirt
@@ -15,36 +15,70 @@ start() {
     # and it needs to be at the front of the forward chain
     iptables -I FORWARD -m physdev --physdev-is-bridged -j ACCEPT
+ krb5_conf=/etc/krb5.conf
+    krb5_tab=/etc/libvirt/krb5.tab
+    # retrieve config from local oVirt partition if available
+    #   krb5.conf krb5.tab
+    #   TODO local admin password, ssh server key - what else?
+    ovirt=$(mktemp -d)
+    cfg=$ovirt/config
+    mount -r /dev/disk/by-label/$OVIRT_LABEL $ovirt
+    if [ -e $cfg/krb5.conf ]; then
+      cp -a $cfg/krb5.conf $krb5_conf
+    fi
+    if [ -e $cfg/krb5.tab ]; then
+      cp -a $cfg/krb5.tab $krb5_tab
+    fi
+    if [ -s $krb5_tab ]; then
+      krb5_tab=
+    fi
+
     find_srv ipa tcp
-    if [ -n "$SRV_HOST" -a -n "$SRV_PORT" ]; then
-        krb5_conf=/etc/krb5.conf
-        if [ ! -s $krb5_conf ]; then
-            rm -f $krb5_conf
-            # FIXME this is IPA specific
-            wget -q \
-                http://$SRV_HOST:$SRV_PORT/ipa/config/krb5.ini -O $krb5_conf \
-                || (echo "Failed to get $krb5_conf" && return 1)
+    if [ -n "$SRV_HOST" -a -n "$SRV_PORT" -a ! -s $krb5_conf ]; then
+        rm -f $krb5_conf
+        # FIXME this is IPA specific
+        wget -q \
+            http://$SRV_HOST:$SRV_PORT/ipa/config/krb5.ini -O $krb5_conf
+        if [ $? -ne 0 ]; then
+            echo "Failed to get $krb5_conf"; return 1
+        fi
+        # store config in oVirt partition
+        if [ -e $cfg ]; then
+            mount -o remount,rw $ovirt
+            cp -a $krb5_conf $cfg/krb5.conf
         fi
     else
-        echo "skipping Kerberos configuration, IPA service not available"
+        echo "skipping Kerberos configuration"
     fi
find_srv identify tcp
     if [ -n "$SRV_HOST" -a -n "$SRV_PORT" ]; then
-        krb5_tab=/etc/libvirt/krb5.tab
-        ovirt-awake start $krb5_tab $SRV_HOST $SRV_PORT
+        ovirt-awake start $SRV_HOST $SRV_PORT $krb5_tab
+        if [ $? -ne 0 ]; then
+            echo "ovirt-awake failed"; return 1
+        fi
+        # store config in oVirt partition
+        if [ -n "$krb_tab" -a -e $cfg ]; then
+            mount -o remount,rw $ovirt
+            cp -a $krb5_tab $cfg/krb5.tab
+        fi
     else
-        echo "skipping ovirt-awake, oVirt registration service not available"
+        echo "skipping ovirt-awake, oVirt identify service not available"
     fi
+ # cleanup
+    umount $ovirt && rmdir $ovirt
+
     find_srv collectd tcp
     if [ -n "$SRV_HOST" -a -n "$SRV_PORT" ]; then
         if [ -f $collectd_conf.in ]; then
             collectd_conf=/etc/collectd.conf
             sed -e "s/@COLLECTD_SERVER@/$SRV_HOST/" \
                 -e "s/@COLLECTD_PORT@/$SRV_PORT/" $collectd_conf.in \
-                > $collectd_conf \
-                || (echo "Failed to write $collectd_conf" && return 1)
+                > $collectd_conf
+            if [ $? -ne 0 ]; then
+                echo "Failed to write $collectd_conf"; return 1
+            fi
         fi
     else
         echo "skipping collectd configuration, collectd service not available"
@@ -53,7 +87,7 @@ start() {
case "$1" in
     start)
-        echo -n $"Starting ovirt: "
+        printf "Starting ovirt: "
{
             start
diff --git a/ovirt-managed-node/src/scripts/ovirt-awake b/ovirt-managed-node/src/scripts/ovirt-awake
index 38d405e..fc03c9a 100755
--- a/ovirt-managed-node/src/scripts/ovirt-awake
+++ b/ovirt-managed-node/src/scripts/ovirt-awake
@@ -63,10 +63,10 @@ start () {
KEYTAB=`echo $REPLY | awk '{ print $2 }'` - if [ -n $KEYTAB ]; then
+            if [ -n "$KEYTAB" -a -n "$KEYTAB_FILE" ]; then
                 echo "Retrieving keytab: '$KEYTAB'"
- wget -q $KEYTAB --output-document=$KEYTAB_FILE
+                wget -q "$KEYTAB" --output-document="$KEYTAB_FILE"
             else
                 echo "No keytab to retrieve"
             fi
@@ -84,9 +84,9 @@ start () {
case "$1" in
     start)
-        KEYTAB_FILE=$2
-        SRV_HOST=$3
-        SRV_PORT=$4
+        SRV_HOST=$2
+        SRV_PORT=$3
+        KEYTAB_FILE=$4
         start
         RETVAL=$?
     ;;
diff --git a/ovirt-managed-node/src/scripts/ovirt-early b/ovirt-managed-node/src/scripts/ovirt-early
index aa0a49c..dada566 100755
--- a/ovirt-managed-node/src/scripts/ovirt-early
+++ b/ovirt-managed-node/src/scripts/ovirt-early
@@ -10,6 +10,9 @@
 . /etc/init.d/functions
 . /etc/init.d/ovirt-functions
+# size of the oVirt partition in megabytes
+OVIRT_SIZE=64
+
 configure_from_network() {
     DEVICE=$1
     if [ -n "$DEVICE" ]; then
@@ -51,21 +54,250 @@ configure_from_network() {
     printf "default config applied."
 }
+# find_disk $bus $serial
+find_disk() {
+    local bus=$1
+    local serial=$2
+    local live=$(readlink -e /dev/live)
+    live=${live%[1-4]}
+    for d in /dev/sd?; do
+      eval $(udevinfo --query env --name $d)
+      if [ "$ID_BUS" = "$bus" ]; then
+        if [ -z "$serial" -o "$ID_SERIAL" = "$serial" ]; then
+          # cannot install LiveOS over itself
+          if [ -n "$live" -a "$d" = "$live" ]; then
+            return 2
+          else
+            echo $d
+            return 0
+          fi
+        fi
+      fi
+    done
+    return 1
+}
+
+# local_install $local_os $target
+#  local_os - 1=install LiveOS and boot loader
+#             0=initialize oVirt partition only
+#  target - target disk to hold the oVirt partition
+#           usb|scsi[:serial#]
+#
+# oVirt partition is a local primary ext2 partition, labeled OVIRT,
+# which contains:
+#  /config - local oVirt configuration (ktab, local admin password)
+#  /boot - boot loader, kernel and initramfs
+#  /LiveOS - oVirt Node compressed livecd image
+
+local_install() {
+    local local_os=$1
+    local target=$2
+    local disk
+    local part
+    local live_dev=/dev/live
+    if [ ! -e $live_dev ]; then
+      # PXE boot
+      live_dev=/dev/loop0
+    fi
+    local ovirt_part=$(readlink -e /dev/disk/by-label/$OVIRT_LABEL)
+    local ovirt_disk=${ovirt_part%[1-4]}
+    if [ "$ovirt_disk" = "$ovirt_part" ]; then
+      ovirt_disk=
+    fi
+    if [ -z "$target" ]; then
+        live_part=$(readlink -e $live_dev)
+        if [ -z "$ovirt_disk" -o "$ovirt_part" = "$live_part" ]; then
+          return 1
+        fi
+        mode=update
+        disk=$ovirt_disk
+        part=$ovirt_part
+    else
+      case "$target" in
+          scsi*)
+          bus=scsi
+          serial=${target#scsi:}
+          ;;
+          usb*)
+          bus=usb
+          serial=${target#usb:}
+          ;;
+          *)
+          return 1
+          ;;
+      esac
+      if [ "$serial" = "$bus" ]; then
+        serial=
+      fi
+      disk=$(find_disk $bus $serial)
+      rc=$?
+      if [ $rc -eq 2 ]; then
+        echo "target '$target' is the boot disk, aborting install"
+        return 1
+      elif [ $rc -ne 0 ]; then
+        echo "target disk '$target' not found"
+        return 1
+      fi
+      mode=install
+      if [ -n "$ovirt_disk" ]; then
+        if [ "$disk" = "$ovirt_disk" ]; then
+          # target disk contains oVirt partition, select it for update
+          # TODO force install option
+          mode=update
+          part=$ovirt_part
+        else
+          # remove label from oVirt partition, there can be only one
+          e2label $ovirt_part ""
+        fi
+      fi
+      if [ "$mode" = "install" ]; then
+        printf "installing $disk..." | tee /dev/console
+        dd if=/dev/zero bs=4096 count=1 of=$disk \
+        && parted -s $disk mklabel msdos \
+        && parted -s $disk mkpart primary ext2 0.0 $OVIRT_SIZE \
+        && partprobe -s $disk
+        if [ $? -ne 0 ]; then
+          echo "$disk partition creation failed"; return 1
+        fi
+        part=${disk}1
+        udevsettle
+        mkfs.ext2 -L $OVIRT_LABEL $part \
+        && tune2fs -c0 -i0 $part
+        if [ $? -ne 0 ]; then
+          echo "$disk ext2 creation failed"; return 1
+        fi
+        # update by-label link manually, mkfs won't trigger udev
+ls -l /dev/disk
+        mkdir -p /dev/disk/by-label
+ls -l /dev/disk/by-label

Are the two ls lines above missing indentation?

+        ln -sf $part /dev/disk/by-label/$OVIRT_LABEL
+      fi
+    fi
+
+    if [ "$mode" = "update" ]; then
+      printf "updating $ovirt_disk..." | tee /dev/console
+    fi
+    ovirt=$(mktemp -d)
+    live=$(mktemp -d)
+    mount -r $live_dev $live
+    if [ $? -ne 0 ]; then
+      echo "source image mount failed"; return 1
+    fi
+    mount $part $ovirt
+    if [ $? -ne 0 ]; then
+      echo "target mount failed"; return 1
+    fi
+    mkdir -p $ovirt/config
+    # update local config using embedded in livecd image
+    # TODO admin tool for adding /config into livecd image
+    if [ -e $live/config/krb5.conf ]; then
+      cp -a $live/config/krb5.conf $ovirt/config \
+      || echo "krb5.conf copy failed"
+    fi
+    if [ -e $live/config/krb5.tab ]; then
+      cp -a $live/config/krb5.tab $ovirt/config \
+      || echo "krb5.tab copy failed"
+    fi

Nice. I hadn't considered this as a feature. We might also want to think about using this to distribute a site-wide SSH key for root user as well as /etc/passwd and /etc/shadow files.

+    if [ $local_os = 0 ]; then
+      # config update only, cleanup and continue booting
+      umount $ovirt && rmdir $ovirt
+      umount $live && rmdir $live
+    else
+      # install oVirt Node image for local boot
+      if [ -e "$live/syslinux" ]; then
+        syslinux=syslinux
+      elif [ -e "$live/isolinux" ]; then
+        syslinux=isolinux
+      else
+        syslinux=
+      fi
+      rm -rf $ovirt/boot
+      rm -rf $ovirt/LiveOS
+      mkdir -p $ovirt/boot/grub
+      mkdir -p $ovirt/LiveOS
+      cp -a $live/LiveOS/squashfs.img $ovirt/LiveOS \
+      && cp -a $live/$syslinux/initrd0.img $ovirt/boot \
+      && cp -a $live/$syslinux/vmlinuz0 $ovirt/boot
+      if [ $? -ne 0 ]; then
+        echo "image copy failed"; return 1
+      fi
+      part_num=$(( ${part#$disk} - 1 ))
+      cat > $ovirt/boot/grub/grub.conf <<EOF
+default=0
+timeout=2
+hiddenmenu
+title oVirt Managed Node
+    root (hd0,$part_num)
+    kernel /boot/vmlinuz0 ro root=LABEL=OVIRT roottypefs=ext2 liveimg
+    initrd /boot/initrd0.img
+EOF

Ian will need to add to this patch later for some serial console stuff probably.

+      grub-install --root-directory=$ovirt $disk >&2
+      if [ $? -ne 0 ]; then
+        echo "boot loader install failed"; return 1
+      fi
+      # remove 1.5 stages we don't need
+      find $ovirt/boot/grub -name '*_stage1_5' ! -name e2fs_stage1_5 \
+        -exec rm {} \;
+      # FIXME avoid reboot loops
+      # - cobbler: pxe-only-once, needs xmlrpc to cobbler server
+      # - plain dnsmasq with tftp: ?
+      # temp. workaround:
+      printf "oVirt local installation finished, press Enter to reboot." \
+        > /dev/console
+      read key
+      if [ "$key" = "debug" ]; then
+        sh > /dev/console 2>&1
+      fi
+      reboot
+    fi
+}
+
 start() {
-    # find boot interface from cmdline
-    # IPAPPEND 2 in pxelinux.cfg appends e.g. BOOTIF=01-00-16-3e-12-34-57
-    BOOTIF=
+    # oVirt boot parameters
+    #   BOOTIF=<MAC> (appended by pxelinux)
+    #   ovirt_init=usb|scsi[:serial#]
+    #   ovirt_local_boot
+
+    #   BOOTIF=<MAC> (appended by pxelinux)

The BOOTIF line is in there twice.

+    # network boot interface is assumed to be on management network where
+    # oVirt Server is reachable
+    # IPAPPEND 2 in pxelinux.cfg appends MAC address of the booting node
+    # e.g. BOOTIF=01-00-16-3e-12-34-57
+    bootif=
+
+    #   ovirt_init=usb|scsi[:serial#]
+    # local installation target disk
+    # usb|scsi - select disk type, as reported by udev ID_BUS
+    # serial# - select exact disk using serial number, as reported by
+    #           udev ID_SERIAL
+    # e.g. ovirt_init=usb:Generic_STORAGE_DEVICE_0000145418-0:0
+    target=
+
+    #   ovirt_local_boot
+    # install/update oVirt Node image on the local installation target disk
+    local_os=0
+
     for i in $(cat /proc/cmdline); do
         case $i in
             BOOTIF=??-??-??-??-??-??-??)
-            i=${i/#BOOTIF=??-/}
-            BOOTMAC=${i//-/:}
-            BOOTIF=$(grep -l $BOOTMAC /sys/class/net/eth*/address|rev|cut -d/ -f2|rev)
+            i=${i#BOOTIF=??-}
+            bootif=$(grep -il $(echo $i|sed 's/-/:/g') /sys/class/net/eth*/address|rev|cut -d/ -f2|rev)
+            ;;
+            ovirt_init=*)
+            target=${i#ovirt_init=}
+            ;;
+            ovirt_local_boot)
+            local_os=1
             ;;
         esac
     done
- configure_from_network $BOOTIF
+    # TODO pass extra kernel params from /proc/cmdline e.g. console=...
+    set -x
+    local_install $local_os $target
+    set +x
+    configure_from_network $bootif
# find all of the partitions on the system @@ -96,7 +328,7 @@ start() { case "$1" in
     start)
-        echo -n $"Starting ovirt-early: "
+        printf "Starting ovirt-early: "
{
             start
diff --git a/ovirt-managed-node/src/scripts/ovirt-functions b/ovirt-managed-node/src/scripts/ovirt-functions
index 5b530f7..083e13d 100644
--- a/ovirt-managed-node/src/scripts/ovirt-functions
+++ b/ovirt-managed-node/src/scripts/ovirt-functions
@@ -2,6 +2,9 @@
OVIRT_LOGFILE=/var/log/ovirt.log +# label of the oVirt partition
+OVIRT_LABEL=OVIRT
+
 find_srv()
 {
     local dnsreply
diff --git a/ovirt-managed-node/src/scripts/ovirt-post b/ovirt-managed-node/src/scripts/ovirt-post
index ecc6ff4..1267f40 100755
--- a/ovirt-managed-node/src/scripts/ovirt-post
+++ b/ovirt-managed-node/src/scripts/ovirt-post
@@ -22,7 +22,7 @@ start() {
case "$1" in
     start)
-        echo -n $"Starting ovirt-post: "
+        printf "Starting ovirt-post: "
{
             start

--
|=-        Red Hat, Engineering, Emerging Technologies, Boston        -=|
|=-                     Email: pmyers redhat com                      -=|
|=-         Office: +1 412 474 3552   Mobile: +1 703 362 9622         -=|
|=- GnuPG: E65E4F3D 88F9 F1C9 C2F3 1303 01FE 817C C5D2 8B91 E65E 4F3D -=|


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]