[Linux-cluster] Snapshotting GFS and freezing

[Ray Van Dolson]

On Wed, Jan 20, 2010 at 06:14:02AM -0800, Dirk H. Schulz wrote:
> Pasi Kärkkäinen schrieb:
> > On Sun, Jan 17, 2010 at 02:47:00PM +0100, Dirk H. Schulz wrote:
> >   
> >> Ray Van Dolson schrieb:
> >>     
> >>>> Hmm.. so snapshots with CLVM are possible nowadays?
> >>>>         
> >>> No....
> >>>
> >>> RH has stated (recently on this list) that patches exist to do it, but
> >>> it hasn't been a high enough priority for them to complete the work to
> >>> the point where it could be distributed to customers.
> >>>  
> >>>       
> >> Okay then, how do you people out there with clusters that run virtual 
> >> machines with live migration do backups of the virtual machines?
> >> I surely do not want to shut down the virtual machine to be able to copy 
> >> the image safely away if I have live migration available.
> >>
> >> At the moment there is only one way I can see. PLEASE prove me to be wrong.
> >> Searching in RedHat's documentation I found that the problem is that lvm 
> >> snapshots need exclusively allocated logical volumes. So I think the 
> >> following
> >> should be technically possible:
> >> 0. Start environment: the logical volume containing the images of the 
> >> virtual machines uses gfs and is mounted on all relevant cluster nodes 
> >> since VMs are running on several cluster nodes.
> >> 1. All VMs have to be migrated to one of the cluster nodes
> >> 2. On all other nodes, the gfs volume is unmounted
> >> 3. On the remaining node (where all VMs now run) the logical volume is 
> >> bound exclusively with "lvchange -aey LOGICALVOLUME"
> >>    (I hope this is possible without deactivating it first)
> >> 4. Now GFS on this volume is frozen: "gfs_tool freeze 
> >> /mountpoint/of/local/volume"
> >>     
> >
> > Before freezing the GFS you should make sure the VMs are in consistent
> > state, and the VMs have flushed their caches/buffers/disks.
> >   
> That means calling  
>         sync
>         echo 3 > /proc/sys/vm/drop_caches
> inside the VM, right? Or is there anything more to do to flush everything?
> 
> Dirk

I would think the best way would be to actually pause the VM.  If you
snapshot a VM while it's running, even with the above, how can you
guarantee an application won't do something on the VM right as the
snapshot occurs?

Ray