[Virtio-fs] Kudos: productivity boost using virtio-fs

[Vivek Goyal]

On Thu, Oct 01, 2020 at 09:12:17AM -0500, Harry G. Coin wrote:
> 
> On 10/1/20 8:51 AM, Dr. David Alan Gilbert wrote:
> > * Harry G. Coin (hgcoin at gmail.com) wrote:
> >> Though it's likely been written before:
> >>
> >> Virtio-fs is a wonderful productivity boost in development environments
> >> in which unrelated deadlocks, crashes and hard-lockups happen.   Why? 
> >> Because the underlying file system is never corrupted (though individual
> >> file contents may be, the file system itself is protected).   For
> >> example, using btrfs' snapshot ability in the underlying file system and
> >> comparing 'before and after' crashes creates a powerful debugging tool.
> > Thanks!
> >
> > I'm curious, can you describe a bit more about how you're using it with
> > btrfs - I don't think we've had anyone describe that before.
> > (We mostly use it with overlayfs via the various container tools).
> >
> > Dave
> 
> The 3 host chassis I've set up so far to work with this has a  btrfs
> file system in a raid-1 setup.  Btrfs does a good job with xattr support
> which virtio-fs can be set to use well.
> 
> I have several vm's running on each chassis, each using virtio-fs.  On
> the host, there is a sub-volume for each vm.  The vms  use kernel
> booting, and huge pages.  So, on the bare-metal host, a simple 'btrfs su
> snapshot <vm sub volume> <snap name> generates an 'almost free' (from a
> resource perspective) look at the entire vm.  Various risky experimental
> things go on in the vm, which could lead to file system corruption owing
> to deadlocks and hard freezes and race conditions.  However with
> virtio-fs, when that happens, all I need to do on the host is make a
> another snapshot of the 'frozen' vm, stop the vm.  Then I can compare
> the exact state of the files at the time of the freeze with the 'known
> good' versions at the snapshot time.   Delete the 'frozen' state
> snapshot, make a new  snapshot of the known good snapshot, reboot the vm
> and the dev cycle can continue. 

IIUC, using virtio-fs saved one reboot of the host. Otherwise
one can just run keep one known good copy and run workload on 
another copy. Once system deadlocks, reboot and compare good and
bad copy? Did I miss the point completely.

> 
> Even if the cause of the freeze wasn't evident in the file system
> deltas, knowing there isn't corruption in the underlying fs is a big
> time saver.
> 
> I expect when dax makes it out of the development bubble this
> configuration will be a very high performance, easy to maintain, high
> security arrangement.  Much harder to 'break out' of a vm than the
> various container options.

DAX should be exciting. Looking forward to it.

One restriction right now it will put is
that don't share the directory with other clients or host. Because
if they truncate the file (which has been mapped inside guest), and
guest accesses that after truncation, then we don't have mechanism
to handle error and guest spins forever.

Trying to solve that issue in KVM but it does not seem to be a trivial
fix.

Thanks
Vivek