[Linux-cachefs] Fedora kernel testing

[Daire Byrne]

David,

----- "David Howells" <dhowells at redhat.com> wrote:

> Daire Byrne <Daire.Byrne at framestore.com> wrote:
> > This would be really cool for the VPN/homedir application. I have found that 
> > trying to run your homedir over a VPN results in very slow application load 
> > times simply because when searching PATHs (e.g. Maya, Python etc.) apps will 
> > make many many open() calls most of which simply return ENOENT (No such
> > file or directory). If we could cache the entries of the dir subsequent file
> > lookups would not need to go to the network.
> 
> Caching negative results would certainly be possible, provided you
> can determine that the directory has been updated when a previously
> negative entry has been instantiated.  This can probably be done reasonably by
> checking the mtime and ctime on the dir.

Ah yes - that makes sense. So you would do lots of opens() and access() calls
the first time on non-existent files and then subsequently only read from the
cache? But wouldn't you still need to check the parent dir ctime/mtime from the
network for every call to ensure nothing in those dirs has changed? Perhaps 
you only want to do this periodically for better performance/less net I/O?
 
> > There are even more access() calls to non-existent files than open() calls 
> > which really slows things down a lot. Does it make sense to cache the perms 
> > of files in a dir too?
> 
> Ummm...  As I said above, negative results due to non-existence should be
> reasonably easy.  We do this with the dentry cache after all.
> 
> However, negative results due to permissions failure is harder to deal
> with.  I don't know that NFS4 will notify you if the annotations on a file
> change, though you can poll the ctime.  Also, this depends on local factors,
> such as the callers credentials, and so you can have several results for a
> single file.
> 
> We could certainly cache the mode/ACL of a file and attempt to make
> the check locally - indeed this would be required for disconnected operation -
> but the final authority still has to be the server.

It seems to me that we would want some more mount options so that you can 
decide at mount time how secure or accurate you want the cached mountpoint to 
be at the expense of performance.

> > Sorry, I wasn't very clear. I'm interested in testing the NFS cache when we
> > have a single common QCOW2 disk image accessed by many (100+) clients but all
> > writes simply go back to a separate QCOW2 images (which QCOW2 supports). I
> > suppose this is the VM equivalent of caching a "diskless" network boot Linux
> > distro except the image is a single file instead of many files. Again knowing
> > that the read-only master image never changes it would be good if one went to
> > the cache before doing any network lookups. We *could* just copy the entire
> > image locally to the machine each time but obviously getting cachefilesd to
> > manage it automatically is more elegant. The performance may be worse though
> > if the PAGEs get written randomly out of order in the corresponding local
> > cache file.
> 
> Is this what you're thinking of:
> 
>  (1) A base image for a virtual disk is made available on an NFS
> server.
> 
>  (2) When clients want to read pages, they check fscache, and if that
> doesn't have a copy of the page, the page is fetched from the server.  If
> fscache does have a copy, it's fetched from there.
> 
>  (3) When a client alters a page, that alteration is stored to fscache
> only; it is not sent back to the server.  Future retrievals of the page
> will then obtain it from fscache, and not the server.

Pretty much, expect that I think it should be reasonable to write directly back to 
NFS to the separate COW image without any caching (and is a requirement for live 
migration anyway). I would think that the read/write ratio of a Linux VM image is 
such that the writes will generate much less load on an NFS server than reads 
would (without NFS caching).

Thanks again for your time and effort.

Daire