[libvirt] a script for libvirt-llxc + systemd on Fedora18

[Kamezawa Hiroyuki]

Hi.
At playing libvirt-lxc on Fedora18, I found that the user needs
some workarounds to run /sbin/init as the root process of container.

With Fedora15, I found this https://gist.github.com/peo3/1142202.
And I know virt-sandbox-service have some clever way to understand RPM
to make domain running with a service.

Here is a script I wrote for F18 inspired by above two. Creating
LXC domain running systemd as /init program. I'd like to get
some feedback....and ...can't this kind of scripts be maintained by some
virt-tools project ? 

Anyway, playing with lxc is fun. I'm glad if someone makes this script better :)
==
A user can create a lxc container by

# ./lxc_systemd_setup.py -r -n Test
  This will generate my.xml and lxc file tree on /opt/lxc/Test

# virsh -c lxc:/// define my.xml

# virsh -c lxc:/// start my.xml

This domain has following characteristics
  - /usr is bind mounted to host's /usr
  - all files other than files in /usr are copied
  - systemd and pam modules are tweaked a bit
  - eth0 is up by dhcp.
  - systemd, rsyslog, sshd is running.

Users can add rpm package as
# ./lxc_systemd_setup.py -r -n Test -a httpd

some options may not work because of my bad skills.

==lxc_systemd_setup.py==
#!/usr/bin/python
#
# A scripts for creating LXC domain where some daemons are running under systemd.
# Tested with Fedora18, running systemd, rsyslog, sshd in a container at default.
#
# Most config files, including all security settings as passwd, selinux,
# ssl/pki files are copied from the host. So, please fix them before
# running a domain.
#
# New root dir will be /opt/lxc/<domain name> at default and domain XML
# definition will be saved as my.xml
#
# All instllation under /usr is shared among containres/host, binaries are
# not copied at all. /etc and /var are copied. So, you can adjust config files
# under container as you like.
#
# for easy creation, run
# # lxc_systemd_setup.py -n Test -r
# # virsh -c lxc:/// define my.xml
# # virsh -c lxc:/// start Test --console
# 
# This will build a lxc root filesystem under /opt/lxc/Test and copy
# required files under it. /usr will be shared with the host.
#
# to add some packages, pass package name with '-a' option. But this script doesn't
# handle dependency of RPM, at all. please take care of it.
#
# for running httpd.
# # lxc_systemd_setup.py -n Test -r -a httpd
# _and_ you need to fix hostname lookup problem to run httpd. i.e. you may need to fix
# some files under /etc....
#
# please see
# # lxc_systemd_setup.py --help
# for other options. some may not work ;)
#
import sys, os, shutil
from subprocess import Popen, PIPE
from optparse import OptionParser,OptionGroup
import re
import rpm

parser = OptionParser()

parser.add_option('-p', '--prefix', action='store', default='/opt/lxc',
            type='string', dest='pathname', help='prefix of guest root path')

parser.add_option('-n', '--name', action='store', default='_unknown',
            type='string', dest='domain_name', help='name of domain')

parser.add_option('-o', '--out', action='store', default='my.xml',
            type='string', dest='def_file', help='name of generated xml def')

parser.add_option('-r', '--renew', action='store_true', default=False,
            dest='renew_tree', help='delete existing tree if exists')

parser.add_option('-D', '--destroy', action='store_true', default=False,
            dest='destroy_tree', help='destroy existing tree and quit')

parser.add_option('-f', '--force', action='store_true', default=False,
            dest='force', help='update all files without checking timestamp')

parser.add_option('-a','--add-packages', action='append',
            type='string', dest='package_list', help='copy package config to container')

parser.add_option('-s','--skip-packages', action='append',
            type='string', dest='skip_list', help='skip packages')

parser.add_option('-m','--memory', action='store', default='1024000',
            type='str', dest='memory', help='memory size of domain')

parser.add_option('-H','--hostname', action='store', default='lxc',
            type='str', dest='hostname', help='hostname of domain')

(options, args) = parser.parse_args(sys.argv)


#
# Utility functions.
# 

#
# Remove all files under domain ROOT.
# 
def destroy_all(path) :
    if (not os.path.exists(path)) :
        return
    shutil.rmtree(path, ignore_errors=True)
#
# Check a file in a container is newer than hosts.
#
def file_is_newer(a, b) :
    time_a = os.stat(a).st_mtime
    time_b = os.stat(b).st_mtime
    return a > b
#
# Check Host's distro.
#
def check_version() :
    useRPM=True
    version="unknown"
    if (os.path.exists('/etc/redhat-release')) :
        with open('/etc/redhat-release') as f:
            version_string = f.readline()
        
    if (re.match("Fedora release 18.+$", version_string)) :
        version = "Fedora18"
    else :
        useRPM=False
    return (version, useRPM)


#
# directories created at domain creation (tested with Fedora 18)
#

ROOTDIR= options.pathname + "/" + options.domain_name

class InstallInfo :
    def __init__(self) :
        self.DIRS = []
        self.BINDDIRS = []
        self.SYMLINKS=[]
        self.PACKAGES = []
        self.FILES = []
        self.MERGED = []

    def add_dirs(self, x) :
        if (isinstance(x, str)) :
            x = [x]
        self.DIRS = self.DIRS + x

    def dirs(self) :
        return self.DIRS

    def add_files(self, x) :
        if (isinstance(x, str)) :
            x = [x]
        self.FILES += x

    def files(self) :
        return self.FILES

    def add_binds(self, x) :
        if (isinstance(x, str)) :
            x = [x]
        self.BINDDIRS = self.BINDDIRS + x
    
    def binds(self) :
        return self.BINDDIRS

    def add_links(self, x) :
        self.SYMLINKS = self.SYMLINKS + x

    def links(self) :
        return self.SYMLINKS

    def add_packages(self, x) :
        self.PACKAGES = self.PACKAGES + x
        
    def packages(self) :
        return self.PACKAGES

    def merge(self) :
        self.DIRS = list(set(self.DIRS))
        self.FILES = list(set(self.FILES))
        self.BINDDIRS = sorted(list(set(self.BINDDIRS)))
        ret = True

        ents =[]
        paths = []
        for ent in self.DIRS :
            ents.append((ent, 'dir', ''))
            paths.append(ent)
        for ent in self.FILES :
            ents.append((ent, 'file', ''))
            paths.append(ent)
        for ent in self.SYMLINKS :
            ents.append((ent[0], 'link', ent[1]))
            paths.append(ent[0])

        if (len(paths) - len(list(set(paths)))) :
            ret = False
        
        self.MERGED = sorted(ents)
        return ret

    def merged(self) :
        return self.MERGED

#
# Gather RPM information and copy config files to proper place.
#
class CopyRPMHandler:
    def __init__(self, name) :
        self.name = name
        self.files = []
        self.service = ""

    def verify(self) :
        ts = rpm.TransactionSet()
        mi = ts.dbMatch('name', self.name)
        if not mi :
            return False
        # get list of files.
        for h in mi :
            myhead = h
            break
        fi = myhead.fiFromHeader()
        for x in fi :
            self.files.append(x[0]);
        self.test_service()
        self.strip_binds()
        return True

    def paths(self) :
        return self.files
    #
    # check all files in RPM which are not under bind-mount.
    #
    def strip_binds(self) :
        # remove all ents under /usr for avoiding copy.
        temp = self.files
        self.files = []
        for file in temp :
            if (file == '') :
                continue
            if (re.match("/usr/.+$", file)) :
                continue
            if (re.match("/bin/.+$", file)) :
                continue
            if (re.match("/lib/.+$", file)) :
                continue
            if (re.match("/sbin/.+$", file)) :
                continue
            if (re.match("/lib64/.+$", file)) :
                continue
            self.files.append(file)
        self.files.sort()
        return

    def test_service(self) :
        for file in self.files :
            if (re.match("/usr/lib/systemd/system.+\.service$", file)) :
                ent.service = file


#
# Functions for workarounds.
#

#
# systemd: create our own basic.target for avoiding some startups.
#
def systemd_tune() :
    #
    # we need to avoid some special services by systemd.
    # modify basic.target and avoid them.
    #
    filename = ROOTDIR + "/etc/systemd/system/basic.target"
    data="""[Unit]
Description=Basic System
Documentation=man:systemd.special(7)
Requires=systemd-tmpfiles-setup.service sockets.target
After=systemd-tmpfiles-setup.service sockets.target
RefuseManualStart=yes
"""
    with open(filename,"w") as f:
        f.write(data)
    #
    # we need getty only with tty1
    #
    os.symlink("/usr/lib/systemd/system/getty at .service",
        ROOTDIR + "/etc/systemd/system/getty.target.wants/getty at tty1.service")
#
# Create ifcfg-eth0 and add service to bring up it.
#
def eth0_service() :
    #
    # /etc/sysconfig/network is generated by annaconda and we cannot
    # find it by rpms.
    # 
    filename = ROOTDIR + "/etc/sysconfig/network"
    shutil.copy("/etc/sysconfig/network", filename);
    #
    # ifconfig setting for eth0
    #
    filename = ROOTDIR + "/etc/sysconfig/network-scripts/ifcfg-eth0"
    data="""DEVICE=eth0
BOOTPROTO=dhcp
ONBOOT=yes
NAME=eth0
TYPE=Ethernet
"""
    with open(filename, "w") as f:
        f.write(data)

    print "Creating %s" % filename
   
    filename = ROOTDIR + "/etc/systemd/system/lxc-eth0.service"
    data="""[Unit]
Before=multi-user.target
Conflicts=shutdown.target
Description=bring up eth0 in this container
[Service]
ExecStart=/usr/sbin/ifup eth0
Type=simple
"""
    with open(filename, "w") as f:
        f.write(data)

    print "Creating %s" % filename
    #
    # Bring up this.
    #
    filename = ROOTDIR + "/etc/systemd/system/basic.target.wants/lxc-eth0.service"
    src = "/etc/systemd/system/lxc-eth0.service"
    os.symlink(src, filename)

#
# Make fstab empty
#
def empty_fstab() :
    filename = ROOTDIR + "/etc/fstab"
    with open(filename, "w") as f:
        f.truncate(0)

#
# in Fedora18, pam's pam_loginuid.so doesn't work under container
# we need to disable it.
#
def pam_tune() :
    pamdir = ROOTDIR + "/etc/pam.d"
    for root, dirs, files in os.walk(pamdir) :
        for path in files :
            path = root + "/" + path
            if (os.path.islink(path)) :
                continue
            data =""
            with open(path) as f:
                for line in  f :
                    if (re.match("^.+pam_loginuid.so.*$", line)) :
                        line = "#" + line
                    data += line
            with open(path, "w") as f:
                f.write(data)
#
# securetty set up for login via system console.
#
def securetty_tune() :
    path = ROOTDIR + "/etc/securetty"
    with open(path, "a") as f:
        f.write("pts/0\n")

#
# set hostname of guest domain.
#
def hostname_modify() :
    path = ROOTDIR + "/etc/hostname"
    with open(path, "w") as f:
        f.write(options.hostname + "\n")

#
# parse memory size.
#
def parse_memory(data) :
    if data[-1] == 'K' :
        x = int(data[0:-1])
        return str(x * 1024)
    elif data[-1] == 'M' :
        x = int(data[0:-1])
        return str(x * 1024 * 1024)
    elif data[-1] == 'G' :
        x = int(data[0:-1])
        return str(x * 1024 * 1024 * 1024)
    else :
        return data

#
# Main routine starts here !
#
version, useRPM = check_version()

if (not useRPM) :
    print 'now, we can handle RPM only'
    exit(1)

info = InstallInfo()

# Build a information.

#
# At first, gather required RPM information and some tweaks for distro.
#
if (version == 'Fedora18') :
    #
    # now, dont'handle yum and rpm info in container, so create fake dirs
    # instead of copying yum info by RPM.
    #
    info.add_dirs(["/etc/yum", "/etc/yum/protected.d",
                    "/etc/yum/pluginconf.d","/etc/yum/vars"])
    #
    # We share /usr between host and guest.
    #
    info.add_binds(["/usr"])
    #
    # For Fedora18, we need following copies of configs packages at least.
    #
    info.add_packages(["filesystem","setup","rpm", "selinux-policy"])
    info.add_packages(["systemd", "dbus", "initscripts","util-linux"])
    info.add_packages(["pam","passwd", "crontabs","kmod","logrotate","rsyslog"])
    info.add_packages(["openssh","openssh-server", "chkconfig","authconfig"])
    info.add_packages(["glibc", "mailcap"])

    # symlink and dirs for systemd
    info.add_links([["/etc/systemd/system/default.target",
                    "/lib/systemd/system/multi-user.target"]])
    info.add_dirs(["/etc/systemd/system/basic.target.wants"])
    info.add_dirs(["/etc/systemd/system/default.target.wants"])
    info.add_dirs(["/etc/systemd/system/getty.target.wants"])
    info.add_dirs(["/etc/systemd/system/multi-user.target.wants"])
    info.add_dirs(["/etc/systemd/system/sockets.target.wants"])
    info.add_dirs(["/etc/systemd/system/sysinit.target.wants"])
    info.add_dirs(["/etc/systemd/system/system-update.target.wants"])

#
# Merge package list
# 
package_names = info.packages()
if (options.package_list) :
    package_names += options.package_list

# Uniq.
package_names = list(set(package_names))

#
# delete unnecessary packages from list.
#
if (options.skip_list) :
    for name in options.skip_list :
       if (name in package_names) :
            package_names.remove(name)

#
# Verify package list (check installation of packages)
#

packages = []
error = False
if (useRPM) :
    for name in package_names :
        ent = CopyRPMHandler(name)
        if (ent.verify()) :
            packages.append(ent)
        else :
            print "Couldn't find a package [%s] in RPM DB." % (name)
            error = True

if (error) :
    exit(1)
#
# Now, we confirmed all RPMS required are installed in the host.
#
service_files = []

#
# Extract dir,symlink,file information from RPMS. Later, we'll copy all
# files other than /usr.
#
for ent in packages :
    for path in ent.paths() :
        if (not os.path.exists(path)) :
            continue
        if (os.path.islink(path)) :
            src = os.readlink(path)
            info.add_links([[path, src]])
        elif (os.path.isfile(path)) :
            info.add_files(path)
        elif (os.path.isdir(path)) :
            info.add_dirs(path)
    if (ent.service != ""):
        service_files.append(ent.service)
#
# Uniq and sort it.
#
if (not info.merge()) :
    print "some confilction of files may happen..."
#
# Check Domain name is passed.
#
if (options.domain_name == '_unknown') :
    print "Guest Domain name must be specified"
    exit(1)

#
# Destroy tree.
#
if (options.destroy_tree) :
    destroy_all(ROOTDIR)
    exit(0)
#
# At first, clear tree if required.
# (*) the scirpt may not work if we don't destroy the tree ....
#
if (os.path.exists(ROOTDIR)) :
    if (options.renew_tree) :
        destroy_all(ROOTDIR)

# Create root dir
try:
   os.mkdir(ROOTDIR)
except:
   print "cannot create root dir %s" % ROOTDIR
   exit(1)


# Ok, make world based on information gathered from RPMS.
for ents in info.merged() :
    guestpath = ROOTDIR + ents[0] 
    try:
        if (ents[1] == 'dir') :
            if (not os.path.exists(guestpath)) :
                print "Creating dir %s" % (guestpath)
                os.makedirs(guestpath)
        elif (ents[1] == 'link') :
            if (not os.path.exists(guestpath)) :
                print "Creating symlink %s => %s" % (guestpath, ents[2])
                os.symlink(ents[2], guestpath)
        elif (ents[1] == 'file') :
            if (options.force or
                not os.path.exists(guestpath) or 
                file_is_newer(ents[0], guestpath)) :
                print "Copyfile %s" % (guestpath)
                shutil.copy(ents[0], guestpath)
    except:
        print "error at creating tree %s" % guestpath
        exit(1)
#
# setup service files if necessary.
#
for file in service_files :
    service = os.path.basename(file)
    p = re.compile('WantedBy=(.+)$')
    for line in open(file, 'r') :
        m = p.match(line)
        if (m) :
            target = m.group(1)
            pathname = ROOTDIR + "/etc/systemd/system/" + target + ".wants/" + service
            print "%s=>%s" % (pathname, file)
            os.symlink(file, pathname)
    
#
# Tweak system settings.
#
if (version == "Fedora18") :
    # diable some services.
    dir = ROOTDIR + "/etc/systemd/system/"
    os.symlink("/dev/null", dir + "sysinit.target")
    os.symlink("/dev/null", dir + "console-shell.service")
    os.symlink("/dev/null", dir + "fedora-readonly.service")
    os.symlink("/dev/null", dir + "fedora-storage-init.service")
    systemd_tune() # modify basic.target etc...
    eth0_service() # bringup eth0 without udev
    empty_fstab()  # make /etc/fstab empty
    pam_tune()     # disable some pam module
    securetty_tune() # add pts/0 to securetty
    hostname_modify()
#
# Generate a Domain Def.
#
domain = r"""
<domain type='lxc'>
  <name>%(NAME)s</name>
  <memory unit='bytes'>%(MEMORY)s</memory>
  <vcpu>1</vcpu>
  <os>
    <type arch='x86_64'>exe</type>
    <init>/sbin/init</init>
  </os>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/libexec/libvirt_lxc</emulator>
    <filesystem type='mount' accessmode='passthrough'>
      <source dir='%(ROOTDIR)s'/>
      <target dir='/'/>
    </filesystem>
    <filesystem type='mount' accessmode='passthrough'>
      <source dir='/usr'/>
      <target dir='/usr'/>
    </filesystem>
    <filesystem type='ram'>
      <source usage='%(MEMORY)s'/>
      <target dir='/tmp'/>
    </filesystem>
    <filesystem type='ram'>
      <source usage='%(MEMORY)s'/>
      <target dir='/dev/shm'/>
    </filesystem>
    <interface type="network">
      <source network="default"/>
    </interface>
    <console type='pty'>
      <target type='lxc' port='0'/>
    </console>
 </devices>
</domain>
""" % {'NAME':options.domain_name,
       'MEMORY': parse_memory(options.memory),
       'ROOTDIR':ROOTDIR}

with open(options.def_file, "w") as f :
    f.write(domain)