[Ovirt-devel] Modeling LVM storage

[Daniel P. Berrange]

On Tue, Sep 16, 2008 at 01:23:15PM -0400, Steve Ofsthun wrote:
> Perry N. Myers wrote:
> >Steve Ofsthun wrote:
> >>Chris Lalancette wrote:
> >>>sseago and I (and variously, other folks) had a somewhat longish 
> >>>conversation on
> >>>IRC today about carving up storage with LVM.  This is the second time 
> >>>we've
> >>>beaten this horse, so hopefully we are somewhat OK now.  The basic 
> >>>idea is that,
> >>>given an iSCSI LUN (and SCSI and FC LUNs in the future), we want to 
> >>>either:
> >>>
> >>>1)  Assign the entire LUN to a guest (this is the way that ovirt 
> >>>works right now)
> >>>2)  Carve up the LUN using LVM, and then hand out individual logical 
> >>>volumes to
> >>>guests.
> >>
> >>How do you plan to distinguish between LVM PVs/LVs created by 
> >>ovirt/libvirt on iscsi LUNs from LVM PVs/LVs created by guest OSes on 
> >>directly connected iscsi LUNs?  Just blindly running pvscan will run 
> >>into all sorts of trouble.
> >
> >All storage is virtualized.  Guests don't access iSCSI LUNs directly, 
> >they access it through libvirt storage API.  So this shouldn't be an 
> >issue.  If we allow guests to directly access iSCSI storage (or other 
> >storage) directly it bypasses our ability to restrict via oVirt 
> >permissions who can access what storage and limit storage usage by quotas.
> 
> I'm not being clear I think.  If libvirt/ovirt allows entire LUNs to be 
> connected as virtual disks to a guest OS, the guest OS can create LVM 
> PVs/LVs on the virtual disks.  These guest PVs/LVs are written directly 
> onto the underlying physical LUNs.  Since these LUNs are also directly 
> visible to the host virtualization software (libvirt, etc), any PVs/LVs 
> created by the guest may be seen by LVM running in the host environment.  
> If this host environment naively runs pvscan, it will see both host created 
> PVs and guest created PVs (on entire LUNS).  The host environment needs to 
> use some method to positively operate on host create PVs/LVs only and 
> ignore guest created PVs/LVs.  This can only happen when entire LUNs are 
> passed to guest OSes.

This also isn't solely an iSCSI issue, its basically a general problem you'll
hit when assigning whole disks to guests, whether their iSCSI, SCSI, IDE,
USB, etc.  At very least you could create a single whole disk partition
and give that to the guest instead - at worst you'd be loosing a sector
or so to the partition table metadata, and the host won't accidentally
see the LVM created by the guest.

If you really want to assign whole disks to guests though, you need to
black/white-list devices in /etc/lvm/lvm.conf, which becomes quite alot
of pain to manage.

Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|