[Ovirt-devel] [PATCH] oVirt Node local installation
Perry N. Myers
pmyers at redhat.com
Fri Aug 8 14:15:11 UTC 2008
Alan Pevec wrote:
> oVirt partition can be initialized on a local flash or disk drive.
> oVirt partition is a local primary ext2 partition, labeled OVIRT,
> which contains:
> /config - local oVirt configuration (ktab, local admin password...)
> /boot - boot loader, kernel and initramfs
> /LiveOS - oVirt Node compressed livecd image
Is the initramfs repeated twice then? Once inside of the LiveCD and once
in /boot? Or am I just reading this wrong...
> oVirt partition is initialized using kernel boot parameters:
> ovirt_init=usb|scsi[:serial#]
> local installation target disk
> usb|scsi - select disk type, as reported by udev ID_BUS
> serial# - select exact disk using serial number, as reported by
> udev ID_SERIAL [1]
> e.g. ovirt_init=usb:Generic_STORAGE_DEVICE_0000145418-0:0
So if you leave off serial#, how does it choose the disk? Does it just
take the first device (i.e. /dev/sda) and start using it?
> ovirt_local_boot
> install/update oVirt Node image on the local installation target disk
> without this parameter oVirt partition is used for storing
> oVirt configuration only
>
> [1] one-liner to list serial#s for all disks in the system:
> for d in /dev/sd?; do eval $(udevinfo --query env --name $d); echo $d $ID_SERIAL;done
Problem with this is it assumes someone has booted linux on the box before
and has access to the above command. I think for the most part people
will not have the serial #s to work with. It's fine to leave this
functionality in, I just wonder how much it would be used.
Other question is... Should we have a sane default in the init scripts?
i.e. even if you don't pass in ovirt_init the init script does the following:
1. Looks to see if there is an existing OVIRT partition and if so mounts
it
2. If one does not exist looks for the first hard disk with free space on
it and creates the OVIRT partition there.
3. If no hard drive space is available then it looks for the first flash
disk with free space on it and creates the OVIRT partition there.
These partitions are for config only, unless the ovirt_local_boot was
passed on the cmdline, in which case the local installation is done.
This way the admin is not forced to always use the ovirt_init parameter on
every machine in the datacenter. The node will do the right thing if the
OVIRT partition is not already there.
Also, steps 2 and 3 can be used to set up local swap if it doesn't already
exist.
Also, if ovirt_local_boot is passed on the command line, the OVIRT
partition should never be placed on the same flash device as the one that
was booted since that device is transient.
Minor comments below...
> Signed-off-by: Alan Pevec <apevec at redhat.com>
> ---
> ovirt-host-creator/common-pkgs.ks | 1 -
> ovirt-managed-node/src/scripts/ovirt | 64 +++++--
> ovirt-managed-node/src/scripts/ovirt-awake | 10 +-
> ovirt-managed-node/src/scripts/ovirt-early | 248 +++++++++++++++++++++++-
> ovirt-managed-node/src/scripts/ovirt-functions | 3 +
> ovirt-managed-node/src/scripts/ovirt-post | 2 +-
> 6 files changed, 298 insertions(+), 30 deletions(-)
>
> diff --git a/ovirt-host-creator/common-pkgs.ks b/ovirt-host-creator/common-pkgs.ks
> index 2d1ad6e..bc08358 100644
> --- a/ovirt-host-creator/common-pkgs.ks
> +++ b/ovirt-host-creator/common-pkgs.ks
> @@ -48,7 +48,6 @@ ovirt-managed-node
> -rhpl
> -kbd
> -usermode
> --grub
> -fedora-logos
> -kpartx
> -dmraid
> diff --git a/ovirt-managed-node/src/scripts/ovirt b/ovirt-managed-node/src/scripts/ovirt
> index 92ed330..b6dcbc6 100755
> --- a/ovirt-managed-node/src/scripts/ovirt
> +++ b/ovirt-managed-node/src/scripts/ovirt
> @@ -15,36 +15,70 @@ start() {
> # and it needs to be at the front of the forward chain
> iptables -I FORWARD -m physdev --physdev-is-bridged -j ACCEPT
>
> + krb5_conf=/etc/krb5.conf
> + krb5_tab=/etc/libvirt/krb5.tab
> + # retrieve config from local oVirt partition if available
> + # krb5.conf krb5.tab
> + # TODO local admin password, ssh server key - what else?
> + ovirt=$(mktemp -d)
> + cfg=$ovirt/config
> + mount -r /dev/disk/by-label/$OVIRT_LABEL $ovirt
> + if [ -e $cfg/krb5.conf ]; then
> + cp -a $cfg/krb5.conf $krb5_conf
> + fi
> + if [ -e $cfg/krb5.tab ]; then
> + cp -a $cfg/krb5.tab $krb5_tab
> + fi
> + if [ -s $krb5_tab ]; then
> + krb5_tab=
> + fi
> +
> find_srv ipa tcp
> - if [ -n "$SRV_HOST" -a -n "$SRV_PORT" ]; then
> - krb5_conf=/etc/krb5.conf
> - if [ ! -s $krb5_conf ]; then
> - rm -f $krb5_conf
> - # FIXME this is IPA specific
> - wget -q \
> - http://$SRV_HOST:$SRV_PORT/ipa/config/krb5.ini -O $krb5_conf \
> - || (echo "Failed to get $krb5_conf" && return 1)
> + if [ -n "$SRV_HOST" -a -n "$SRV_PORT" -a ! -s $krb5_conf ]; then
> + rm -f $krb5_conf
> + # FIXME this is IPA specific
> + wget -q \
> + http://$SRV_HOST:$SRV_PORT/ipa/config/krb5.ini -O $krb5_conf
> + if [ $? -ne 0 ]; then
> + echo "Failed to get $krb5_conf"; return 1
> + fi
> + # store config in oVirt partition
> + if [ -e $cfg ]; then
> + mount -o remount,rw $ovirt
> + cp -a $krb5_conf $cfg/krb5.conf
> fi
> else
> - echo "skipping Kerberos configuration, IPA service not available"
> + echo "skipping Kerberos configuration"
> fi
>
> find_srv identify tcp
> if [ -n "$SRV_HOST" -a -n "$SRV_PORT" ]; then
> - krb5_tab=/etc/libvirt/krb5.tab
> - ovirt-awake start $krb5_tab $SRV_HOST $SRV_PORT
> + ovirt-awake start $SRV_HOST $SRV_PORT $krb5_tab
> + if [ $? -ne 0 ]; then
> + echo "ovirt-awake failed"; return 1
> + fi
> + # store config in oVirt partition
> + if [ -n "$krb_tab" -a -e $cfg ]; then
> + mount -o remount,rw $ovirt
> + cp -a $krb5_tab $cfg/krb5.tab
> + fi
> else
> - echo "skipping ovirt-awake, oVirt registration service not available"
> + echo "skipping ovirt-awake, oVirt identify service not available"
> fi
>
> + # cleanup
> + umount $ovirt && rmdir $ovirt
> +
> find_srv collectd tcp
> if [ -n "$SRV_HOST" -a -n "$SRV_PORT" ]; then
> if [ -f $collectd_conf.in ]; then
> collectd_conf=/etc/collectd.conf
> sed -e "s/@COLLECTD_SERVER@/$SRV_HOST/" \
> -e "s/@COLLECTD_PORT@/$SRV_PORT/" $collectd_conf.in \
> - > $collectd_conf \
> - || (echo "Failed to write $collectd_conf" && return 1)
> + > $collectd_conf
> + if [ $? -ne 0 ]; then
> + echo "Failed to write $collectd_conf"; return 1
> + fi
> fi
> else
> echo "skipping collectd configuration, collectd service not available"
> @@ -53,7 +87,7 @@ start() {
>
> case "$1" in
> start)
> - echo -n $"Starting ovirt: "
> + printf "Starting ovirt: "
>
> {
> start
> diff --git a/ovirt-managed-node/src/scripts/ovirt-awake b/ovirt-managed-node/src/scripts/ovirt-awake
> index 38d405e..fc03c9a 100755
> --- a/ovirt-managed-node/src/scripts/ovirt-awake
> +++ b/ovirt-managed-node/src/scripts/ovirt-awake
> @@ -63,10 +63,10 @@ start () {
>
> KEYTAB=`echo $REPLY | awk '{ print $2 }'`
>
> - if [ -n $KEYTAB ]; then
> + if [ -n "$KEYTAB" -a -n "$KEYTAB_FILE" ]; then
> echo "Retrieving keytab: '$KEYTAB'"
>
> - wget -q $KEYTAB --output-document=$KEYTAB_FILE
> + wget -q "$KEYTAB" --output-document="$KEYTAB_FILE"
> else
> echo "No keytab to retrieve"
> fi
> @@ -84,9 +84,9 @@ start () {
>
> case "$1" in
> start)
> - KEYTAB_FILE=$2
> - SRV_HOST=$3
> - SRV_PORT=$4
> + SRV_HOST=$2
> + SRV_PORT=$3
> + KEYTAB_FILE=$4
> start
> RETVAL=$?
> ;;
> diff --git a/ovirt-managed-node/src/scripts/ovirt-early b/ovirt-managed-node/src/scripts/ovirt-early
> index aa0a49c..dada566 100755
> --- a/ovirt-managed-node/src/scripts/ovirt-early
> +++ b/ovirt-managed-node/src/scripts/ovirt-early
> @@ -10,6 +10,9 @@
> . /etc/init.d/functions
> . /etc/init.d/ovirt-functions
>
> +# size of the oVirt partition in megabytes
> +OVIRT_SIZE=64
> +
> configure_from_network() {
> DEVICE=$1
> if [ -n "$DEVICE" ]; then
> @@ -51,21 +54,250 @@ configure_from_network() {
> printf "default config applied."
> }
>
> +# find_disk $bus $serial
> +find_disk() {
> + local bus=$1
> + local serial=$2
> + local live=$(readlink -e /dev/live)
> + live=${live%[1-4]}
> + for d in /dev/sd?; do
> + eval $(udevinfo --query env --name $d)
> + if [ "$ID_BUS" = "$bus" ]; then
> + if [ -z "$serial" -o "$ID_SERIAL" = "$serial" ]; then
> + # cannot install LiveOS over itself
> + if [ -n "$live" -a "$d" = "$live" ]; then
> + return 2
> + else
> + echo $d
> + return 0
> + fi
> + fi
> + fi
> + done
> + return 1
> +}
> +
> +# local_install $local_os $target
> +# local_os - 1=install LiveOS and boot loader
> +# 0=initialize oVirt partition only
> +# target - target disk to hold the oVirt partition
> +# usb|scsi[:serial#]
> +#
> +# oVirt partition is a local primary ext2 partition, labeled OVIRT,
> +# which contains:
> +# /config - local oVirt configuration (ktab, local admin password)
> +# /boot - boot loader, kernel and initramfs
> +# /LiveOS - oVirt Node compressed livecd image
> +
> +local_install() {
> + local local_os=$1
> + local target=$2
> + local disk
> + local part
> + local live_dev=/dev/live
> + if [ ! -e $live_dev ]; then
> + # PXE boot
> + live_dev=/dev/loop0
> + fi
> + local ovirt_part=$(readlink -e /dev/disk/by-label/$OVIRT_LABEL)
> + local ovirt_disk=${ovirt_part%[1-4]}
> + if [ "$ovirt_disk" = "$ovirt_part" ]; then
> + ovirt_disk=
> + fi
> + if [ -z "$target" ]; then
> + live_part=$(readlink -e $live_dev)
> + if [ -z "$ovirt_disk" -o "$ovirt_part" = "$live_part" ]; then
> + return 1
> + fi
> + mode=update
> + disk=$ovirt_disk
> + part=$ovirt_part
> + else
> + case "$target" in
> + scsi*)
> + bus=scsi
> + serial=${target#scsi:}
> + ;;
> + usb*)
> + bus=usb
> + serial=${target#usb:}
> + ;;
> + *)
> + return 1
> + ;;
> + esac
> + if [ "$serial" = "$bus" ]; then
> + serial=
> + fi
> + disk=$(find_disk $bus $serial)
> + rc=$?
> + if [ $rc -eq 2 ]; then
> + echo "target '$target' is the boot disk, aborting install"
> + return 1
> + elif [ $rc -ne 0 ]; then
> + echo "target disk '$target' not found"
> + return 1
> + fi
> + mode=install
> + if [ -n "$ovirt_disk" ]; then
> + if [ "$disk" = "$ovirt_disk" ]; then
> + # target disk contains oVirt partition, select it for update
> + # TODO force install option
> + mode=update
> + part=$ovirt_part
> + else
> + # remove label from oVirt partition, there can be only one
> + e2label $ovirt_part ""
> + fi
> + fi
> + if [ "$mode" = "install" ]; then
> + printf "installing $disk..." | tee /dev/console
> + dd if=/dev/zero bs=4096 count=1 of=$disk \
> + && parted -s $disk mklabel msdos \
> + && parted -s $disk mkpart primary ext2 0.0 $OVIRT_SIZE \
> + && partprobe -s $disk
> + if [ $? -ne 0 ]; then
> + echo "$disk partition creation failed"; return 1
> + fi
> + part=${disk}1
> + udevsettle
> + mkfs.ext2 -L $OVIRT_LABEL $part \
> + && tune2fs -c0 -i0 $part
> + if [ $? -ne 0 ]; then
> + echo "$disk ext2 creation failed"; return 1
> + fi
> + # update by-label link manually, mkfs won't trigger udev
> +ls -l /dev/disk
> + mkdir -p /dev/disk/by-label
> +ls -l /dev/disk/by-label
Are the two ls lines above missing indentation?
> + ln -sf $part /dev/disk/by-label/$OVIRT_LABEL
> + fi
> + fi
> +
> + if [ "$mode" = "update" ]; then
> + printf "updating $ovirt_disk..." | tee /dev/console
> + fi
> + ovirt=$(mktemp -d)
> + live=$(mktemp -d)
> + mount -r $live_dev $live
> + if [ $? -ne 0 ]; then
> + echo "source image mount failed"; return 1
> + fi
> + mount $part $ovirt
> + if [ $? -ne 0 ]; then
> + echo "target mount failed"; return 1
> + fi
> + mkdir -p $ovirt/config
> + # update local config using embedded in livecd image
> + # TODO admin tool for adding /config into livecd image
> + if [ -e $live/config/krb5.conf ]; then
> + cp -a $live/config/krb5.conf $ovirt/config \
> + || echo "krb5.conf copy failed"
> + fi
> + if [ -e $live/config/krb5.tab ]; then
> + cp -a $live/config/krb5.tab $ovirt/config \
> + || echo "krb5.tab copy failed"
> + fi
Nice. I hadn't considered this as a feature. We might also want to think
about using this to distribute a site-wide SSH key for root user as well
as /etc/passwd and /etc/shadow files.
> + if [ $local_os = 0 ]; then
> + # config update only, cleanup and continue booting
> + umount $ovirt && rmdir $ovirt
> + umount $live && rmdir $live
> + else
> + # install oVirt Node image for local boot
> + if [ -e "$live/syslinux" ]; then
> + syslinux=syslinux
> + elif [ -e "$live/isolinux" ]; then
> + syslinux=isolinux
> + else
> + syslinux=
> + fi
> + rm -rf $ovirt/boot
> + rm -rf $ovirt/LiveOS
> + mkdir -p $ovirt/boot/grub
> + mkdir -p $ovirt/LiveOS
> + cp -a $live/LiveOS/squashfs.img $ovirt/LiveOS \
> + && cp -a $live/$syslinux/initrd0.img $ovirt/boot \
> + && cp -a $live/$syslinux/vmlinuz0 $ovirt/boot
> + if [ $? -ne 0 ]; then
> + echo "image copy failed"; return 1
> + fi
> + part_num=$(( ${part#$disk} - 1 ))
> + cat > $ovirt/boot/grub/grub.conf <<EOF
> +default=0
> +timeout=2
> +hiddenmenu
> +title oVirt Managed Node
> + root (hd0,$part_num)
> + kernel /boot/vmlinuz0 ro root=LABEL=OVIRT roottypefs=ext2 liveimg
> + initrd /boot/initrd0.img
> +EOF
Ian will need to add to this patch later for some serial console stuff
probably.
> + grub-install --root-directory=$ovirt $disk >&2
> + if [ $? -ne 0 ]; then
> + echo "boot loader install failed"; return 1
> + fi
> + # remove 1.5 stages we don't need
> + find $ovirt/boot/grub -name '*_stage1_5' ! -name e2fs_stage1_5 \
> + -exec rm {} \;
> + # FIXME avoid reboot loops
> + # - cobbler: pxe-only-once, needs xmlrpc to cobbler server
> + # - plain dnsmasq with tftp: ?
> + # temp. workaround:
> + printf "oVirt local installation finished, press Enter to reboot." \
> + > /dev/console
> + read key
> + if [ "$key" = "debug" ]; then
> + sh > /dev/console 2>&1
> + fi
> + reboot
> + fi
> +}
> +
> start() {
> - # find boot interface from cmdline
> - # IPAPPEND 2 in pxelinux.cfg appends e.g. BOOTIF=01-00-16-3e-12-34-57
> - BOOTIF=
> + # oVirt boot parameters
> + # BOOTIF=<MAC> (appended by pxelinux)
> + # ovirt_init=usb|scsi[:serial#]
> + # ovirt_local_boot
> +
> + # BOOTIF=<MAC> (appended by pxelinux)
The BOOTIF line is in there twice.
> + # network boot interface is assumed to be on management network where
> + # oVirt Server is reachable
> + # IPAPPEND 2 in pxelinux.cfg appends MAC address of the booting node
> + # e.g. BOOTIF=01-00-16-3e-12-34-57
> + bootif=
> +
> + # ovirt_init=usb|scsi[:serial#]
> + # local installation target disk
> + # usb|scsi - select disk type, as reported by udev ID_BUS
> + # serial# - select exact disk using serial number, as reported by
> + # udev ID_SERIAL
> + # e.g. ovirt_init=usb:Generic_STORAGE_DEVICE_0000145418-0:0
> + target=
> +
> + # ovirt_local_boot
> + # install/update oVirt Node image on the local installation target disk
> + local_os=0
> +
> for i in $(cat /proc/cmdline); do
> case $i in
> BOOTIF=??-??-??-??-??-??-??)
> - i=${i/#BOOTIF=??-/}
> - BOOTMAC=${i//-/:}
> - BOOTIF=$(grep -l $BOOTMAC /sys/class/net/eth*/address|rev|cut -d/ -f2|rev)
> + i=${i#BOOTIF=??-}
> + bootif=$(grep -il $(echo $i|sed 's/-/:/g') /sys/class/net/eth*/address|rev|cut -d/ -f2|rev)
> + ;;
> + ovirt_init=*)
> + target=${i#ovirt_init=}
> + ;;
> + ovirt_local_boot)
> + local_os=1
> ;;
> esac
> done
>
> - configure_from_network $BOOTIF
> + # TODO pass extra kernel params from /proc/cmdline e.g. console=...
> + set -x
> + local_install $local_os $target
> + set +x
> + configure_from_network $bootif
>
> # find all of the partitions on the system
>
> @@ -96,7 +328,7 @@ start() {
>
> case "$1" in
> start)
> - echo -n $"Starting ovirt-early: "
> + printf "Starting ovirt-early: "
>
> {
> start
> diff --git a/ovirt-managed-node/src/scripts/ovirt-functions b/ovirt-managed-node/src/scripts/ovirt-functions
> index 5b530f7..083e13d 100644
> --- a/ovirt-managed-node/src/scripts/ovirt-functions
> +++ b/ovirt-managed-node/src/scripts/ovirt-functions
> @@ -2,6 +2,9 @@
>
> OVIRT_LOGFILE=/var/log/ovirt.log
>
> +# label of the oVirt partition
> +OVIRT_LABEL=OVIRT
> +
> find_srv()
> {
> local dnsreply
> diff --git a/ovirt-managed-node/src/scripts/ovirt-post b/ovirt-managed-node/src/scripts/ovirt-post
> index ecc6ff4..1267f40 100755
> --- a/ovirt-managed-node/src/scripts/ovirt-post
> +++ b/ovirt-managed-node/src/scripts/ovirt-post
> @@ -22,7 +22,7 @@ start() {
>
> case "$1" in
> start)
> - echo -n $"Starting ovirt-post: "
> + printf "Starting ovirt-post: "
>
> {
> start
--
|=- Red Hat, Engineering, Emerging Technologies, Boston -=|
|=- Email: pmyers at redhat.com -=|
|=- Office: +1 412 474 3552 Mobile: +1 703 362 9622 -=|
|=- GnuPG: E65E4F3D 88F9 F1C9 C2F3 1303 01FE 817C C5D2 8B91 E65E 4F3D -=|
More information about the ovirt-devel
mailing list