[libvirt-users] sharing read-only LOCAL disks among KVM guests

[Daniel P. Berrange]

On Fri, Sep 21, 2012 at 09:48:58PM +1000, Amos Shapira wrote:
> On 21 September 2012 21:40, Daniel P. Berrange <berrange at redhat.com> wrote:
> 
> > On Fri, Sep 21, 2012 at 09:34:02PM +1000, Amos Shapira wrote:
> > > On 21 September 2012 21:30, Daniel P. Berrange <berrange at redhat.com>
> > wrote:
> > >
> > > > On Fri, Sep 21, 2012 at 09:22:24PM +1000, Amos Shapira wrote:
> > > > > Hello,
> > > > >
> > > > > Is it possible to share a single host's Logical Volume among multiple
> > > > local
> > > > > KVM guests which mount it read-only?
> > > > >
> > > > > I'm asking this because I have an idea to run multiple idential KVM
> > > > guests
> > > > > (they all have exactly the same software installed on them), booting
> > them
> > > > > from a shared local Logical Volume read-only root file system, or
> > > > > alternatively let them share the bulk of the software (/usr, /opt,
> > /lib)
> > > > > from a common KVM host Logical volume.
> > > > >
> > > > > Is this possible? All my searches so far failed to turn up anything
> > like
> > > > > this.
> > > >
> > > > From the libvirt POV, there's nothing much todo except add <readonly/>
> > > > inside the <disk> element. This will ensure QEMU only gets given read
> > > > permission on the disk backend. The important thing is to then make
> > sure
> > > > your guests actually mount the filesystem with the readonly flag.
> > > >
> > > > > Would it be possible using qcow2 instead of raw LV? If so - would it
> > be
> > > > > worth the performance hit of switching from LV to qcow2?
> > > >
> > > > The type of backend storage doesn't really affect things if the
> > > > disk is fully readonly.
> > > >
> > >
> > > Thanks very much! That's very helpful to know.
> > >
> > > Is this something that someone has already done before (booting multiple
> > > KVM guests from shared read-only root file system) or am I on my own with
> > > this?
> >
> > I've do it with libvirt-sandbox, but using a different approach. Instead
> > of a shared readonly disk image, I use the plan9 filesystem to pass a
> > chroot area from the host to the guest. Then I have to mount certain
> > areas with tmpfs or other writable filesystems (eg /var, /tmp/, parts
> > of /etc, etc).
> >
> > What you want todo is definitely doable - the hard part is just in making
> > your OS image in a way to works with readonly root. This is typically not
> > to well documented, so involves alot of trial and error at first.
> >
> 
> Thanks. I'll try to make my workplace interested in this and report back
> if/when I have interesting results.
> 
> In general, I'm thinking of replicating the way that diskless NFS clients
> used to boot from a common root file system on an NFS server, where the
> only difference among them is their MAC address, which in turn lets them
> pull a little different information (e.g. host name) from DHCP and use this
> as a basis to mount the private parts of the disk (e.g. /var, /etc?) and
> configuration.
> 
> The other question is whether I am correct in expecting that this could
> improve my disk IO profile, since all the guests would be accessing the
> same host's disk blocks.

The normal KVM recommended practice is to use cache=none and let the
guest OS cache I/O on the basis that you can then directly control
guest memory usage. This is probably the exact opposite of what you
want though. To maximise performance I expect you want to let the host
OS aggressively cache the shared disk image (cache=writethrough/unsafe),
and limit (but not eliminate) the amount of extra caching the guest does

Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|