[Libvir] Concepts in storage management

[Chris Lalancette]

Daniel P. Berrange wrote:
> Second, there is clearly a huge number of storage technologies here and 
> there's no way we'll implement support for all of them in one go. So we 
> need to prioritize getting the conceptual model correct, to allow us to
> incrementally support new types of storage backend.

Yeah, I think that's the right place to start.  As you say, there are 
just too many underlying storage technologies to go after all of them at 
once.

<snip>

> 
> Storage representation
> ======================
> 
> Two core concepts
> 
>  - Volume
>     - a chunk of storage
>     - assignable to a guest
>     - assignable to a pool
>     - optionally part of a pool
> 
>  - Pool
>     - a chunk of storage
>     - contains free space
>     - allocate to provide volumes
>     - compromised of volumes
> 
> Recursive! 
> 
>   n x Volume -> Pool -> n x Volume 
> 
> Nesting to many levels...

Kind of, though I think there are actually two concepts of Volumes here 
(if I am understanding correctly).  The first concept of volume is "raw 
storage" -> what you assign to a pool.  The second concept is "Volume 
exported for a guest".  I'm not sure that we want to Nest those concepts.

> 
> Do we need an explicit Filesystem concept ?
> 
> Operations
> ==========
> 
> Limited set of operations to perform
> 
>  - List host volumes   (physical attached devices)
>  - List pools          (logical volume groups, partitioned devs, filesystems)
>  - List pool volumes   (dev partitions, LVM logical volumes, files)
> 
>  - Define pool      (eg create directory, or define iSCSI target)
>  - Undefine pool    (delete directory, undefine iSCSI config
>  - Activate pool    (mount NFS volume,  login to iSCSI target)
>  - Deactivate pool  (unmount volume,  logout of iSCSI)
>  - Dump pool XML    (get all the metadata)
>  - Lookup by path
>  - Lookup by UUID
>  - Lookup by name
> 
>  - Create volume    (create a file, allocate a LVM LV, etc)
>  - Destroy volume   (delete a file, deallocate a LVM LV)
>  - Resize volume    (grow or shrink volume)
>  - Copy volume      (copy data between volumes)
>  - Snapshot volume  (snapshot a volume)
>  - Dump volume XML  (get all the metadata)
>  - Lookup by path
>  - Lookup by UUID
>  - Lookup by name
> 
>  http://www.redhat.com/archives/libvir-list/2007-February/msg00010.html
>  http://www.redhat.com/archives/libvir-list/2007-September/msg00119.html
> 
> Do we also need some explicit Filesystem APIs ?

The question I have with all of this is whether it really belongs in 
libvirt at all.  Many of these concepts apply to bare-metal provisioning 
as well; so it might be a good idea to have a separate "libstorage" that 
libvirt links to, and that other tools might use.

> 
> XML description
> ===============
> 
> The horrible recursiveness & specific attributes are all in the XML 
> description for different storage pool / volume types. This is where
> we define things like what physical volume are in a volume group,
> iSCSI server / target names, login details, etc, etc
> 
>   XXX fill in the hard stuff for metadata description here 
> 
> Implementation backends
> =======================
> 
>  - FileSystem/Directory/File  - POSIX APIs
>  - LVM                        - LVM tools, or libLVM 
>  - Disk/partitions            - sysfs / parted
>  - iSCSI                      - sysfs / iscsi utils
>  - ZFS                        - ZFS tools

The problem with most of these, as we all know, is that they only have 
command-line utilities, and no corresponding libraries.  That makes it 
difficult for a library like libvirt to support them.  That is, we can 
shell out to the commands, but then we run into a situation where 
different versions of the LVM command, for example, have different 
output.  We have now effectively tied ourselves to a particular version 
of a tool, which is fairly disappointing.  Also, as we have seen with 
xend, spawning external tools to do work makes error reporting far more 
difficult (maybe impossible).

So that leaves us the tough question of what to do here.  Ideally we 
would abstract all of the above tools into libraries, and re-write the 
tools to use those, and then have libvirt use the same.  I'm not sure if 
it is practical, however, to wait that long to do such things.

What I might suggest is a hybrid approach.  Do the initial 
implementation with what we have (namely the command line utilities, 
possibly utilizing rjones "scriplet" concept).  In parallel, make sure 
someone starts on making real libraries of the tools, so that we can 
benefit from that later on.

> Implementation strategy
> =======================
> 
> Should prioritize implementation according to immediate application needs
> 
> Initial goal to support remote guest creation on par with current
> capabilities:
> 
>  - Directory + allocateing raw sparse files
>  - Enumerate existing disks, partitions & LVM volumes
> 

Yep.  As long as we make sure the XML is flexible enough to handle the 
remaining stuff, this seems like a good first place to start.

Chris Lalancette