[Ovirt-devel] Ovirt Host Tasks

[Perry N. Myers]

Daniel P. Berrange wrote:
> On Sun, Mar 16, 2008 at 11:43:02PM -0400, Perry N. Myers wrote:
>> 1. Image Creation:
>>
>> a. Integrate whitelist functionality into the image creation
>>    process to reduce image size.
>> b. Determine the minimal set of files that the host image needs to
>>    construct a baseline whitelist.
> 
> IMHO, a pure whitelist approach is not very maintainable long term. As
> we track new Fedora RPM updates, new files often appear & we'll miss them
> with a pure whitelist. I'd go for a combined white/black list so we can 
> be broadly inclusive and only do fine grained whitelist in specific areas

Agreed.  I just use whitelist instead of white/black list because it's 
shorter to type :)

>> c. Determine minimal set of kernel modules to support wide platform base
>>    (i.e. keep disk, net modules.  remove things like sound, isdn, etc...)
>> d. Construct a repeatable build process so that images can be built
>>    daily for testing.
>> e. Construct a utility that detects the minimal set of kernel modules
>>    for a given platform and generates a config file that can be fed to
>>    the build system for generating hardware platform specific images.
>> f. Construct a method for taking the generic host image and removing all
>>    kernel modules not applicable for a specific platform.
> 
> I'd put e) & f) fairly low on the list, unless someone has a compelling
> short term need to make a *tiny* embedded image for specific hardware.

They're certainly lower on the list (hence e and f) but this may 
eventually be a concern for embedding the HV on hardware platforms 
(integrated flash).  Hardware manufacturers will want to keep costs low, 
which means minimal HV size so their flash is as small as possible.  In 
these cases, I'm sure they'll want customized images that only contain the 
kmods for their hardware.

>> g. Construct a method for taking an image and adding site specific
>>    configuration information to it (IP addresses of Ovirt Mgmt Server and
>>    FreeIPA Server).  This is only necessary to support hosts that require
>>    static IP configuration -and- do not have persistent storage to store
>>    configuration information.  (If we decide to support static IP config
>>    we could make presence of persistent storage a requirement...)
>>
>> The goal is to get disk footprint down to something like 32, 64 or 128MB. 
>>  The base images we build will not be hardware platform specific, so 32MB 
>> might be a long shot.  But after step e) above, we should fit into 32MB.
> 
> 64MB is probably a worthy immediate goal to aim for given that the current 
> live image is about 85 MB & we know there's stuff that can be killed. 

Agreed.

> Again, unless we have compelling hardware use case that requires 32 MB,
> I'd not bother aiming for the 32MB mark in the short-medium term. There's
> other more broadly useful stuff we can do....

Agreed.

>> OEM Pre-Install (will always have flash/disk storage)
>>   DHCP Addressing
>>     Flash/Disk Storage/TPM
>>       Auth credentials can be stored in a file or in TPM
>>   Static Addressing
>>     Flash/Disk Storage/TPM
>>       Auth credentials/IP config/Server IPs can be stored in a file
>>       If TPM is present, can be used to store auth info
>> IT Install (may not have flash or disk)
>>   DHCP Addressing
>>     Flash/Disk Storage/TPM
>>       Auth credentials can be stored in a file or in TPM
>>     No Persistent Storage (CD or PXE Boot)
>>       Auth can be stored in host image (separate image for each host)
>>       (NOTE: Not secure for PXE boot.  Recommend that we only allow
>>       CD boot in this case)
>>   Static Addressing
>>     Flash/Disk Storage/TPM
>>       Auth credentials/IP config/Server IPs can be stored in a file
>>       If TPM is present, can be used to store auth info
>>     No Persistent Storage (CD or PXE Boot)
>>       Auth/IP config/Server IPs can be stored in host image (separate
>>       image for each host)
>>       (NOTE: Not secure for PXE boot as above)
> 
> The 2 PXE boot methods can be secure given a deployment scenario where
> the network admin has a dedicated PXE management LAN for their hosts.
> So PXE traffic would be separated from general guest / user traffic.
> Its probably not viable to mandate separate 'management network' but
> we should document it as a deployment scenario.

Good point.

>> 5. KVM Work
>>
>> * Need to work on support for Live Migration
>> * Need to make sure that power management and suspend/resume of guests
>>   work as expected
> 
>  * dmidecode support so we can expose UUID to guest OS
>  * inter-VM socket family. Like AF_UNIX, but between guests (or perhaps
>    just between guest & host). Call it AF_VIRT or some such. Want stream
>    based protocol, but optionally datagram too. 
> 
>    Some use cases for this:
>      - Fast channel to feed performance stats from guest OS to host
>      - Used to build the cluster fence agent
>      - Other... ?

I'll add these to my working list.

>> 7. Performance Monitoring/Auditing/Health
>>
>> * collectd is used presently to send performance stats to management
>>   server.  Is this the right solution?  I don't have a better one, but
>>   suggestions are appreciated.
> 
> collectd is the best option I know of for flexible, lightweight monitoring
> 
>> * libvirt talks to auditd to send audit info to FreeIPA server
> 
> To clarify, there's no explicit depdancy  chain here. Libvirt would simply
> send data to the audit daemon. The audit daemon can optionally be configured
> to ship data off to FreIPA.

Yeah, I just mention it to make sure that we actually do that 
configuration of the audit daemon :)

>> * Need to create a health monitoring daemon that communicates to mgmt
>>   server:
>>   - Sends heartbeat at configurable interval
>>   - Sends status changes of host (including machine check exceptions,
>>     and other errors)
>>   - Sends VM state changes
> 
> This last point should arguably be supported by libvirt directly.

Ok.

>> 8. Clustering
>>
>> Physical clustering is not particularly useful in and of itself, since the 
>> whole point of what we're trying to do is abstract the applications from 
>> hardware further.
> 
> You fundamentally need at least the fencing part of clustering on the physical
> hosts. Without this you have no guarentee that the host & guests that were
> running on it are actually dead. You need this guarentee before starting the
> guests on a new host.

I agree.  I didn't say that physical clustering is not useful at all, I 
just said it wasn't useful by itself.  The clustering solution for Ovirt 
will require both hardware fences on the physical hosts as well as the 
virtual fences provided by libvirt.

>> Clustering of guests should be done.  The hardware fence would be replaced 
>> by a paravirt driver in the guest that communicates to the host.  If two 
>> machines A and B are clustered and on different physical hardware, A is 
>> the primary and B is backup.  B monitors the application on A.  If A 
>> appears to go down, B uses the PV driver to tell the host that it needs to 
>> kill A.  The host tells libvirt that A needs to be destroyed, and that 
>> command is sent over libvirt comms to the physical host that is hosting A.
> 
> The basic goal here is that the oVirt host should provide a general purpose
> virtual fence device to all guests. The guest admin decides which of their
> guests are in the same cluster & does some config task with the driver to
> set this up. Since oVirt knows what VMs each user owns, it can validate the
> config to ensure the admin isn't trying to cluster someone else's VMs. THe
> driver will then provide the guest admin the ability to have one guest shoot
> another guest in the head securely.  The key is there must be *zero* need
> for the guest admin to actually interact with the host admin. The host image
> should support this all automatically, so guest admin's can setup clustering
> in their guest at will.

Thanks for making that more clear :)

Perry