[Linux-cachefs] FS-Cache/CacheFS mark II

[David Howells]

I've written a new version of FS-Cache and CacheFS. It's not quite complete
yet, but I thought I'd wave it in front of you.

What's changed:

 (*) FS-Cache/general:

     - Readpages now supported.
     - Caches are now "tagged" for easier selection from multiple caches.
     - Entries in one index need not have keys of the same size.
     - Netfs interface simplification - page tokens no longer exist.
     - Cookie definitions now belong to cookies not their parents.
     - Data objects now have definitions.
     - Objects can have arbitrary types other than index and data file.
     - Any object can have child objects.
     - Any non-index object can have data.
     - Cookies aren't allocated for any uncached non-index object.
     - Operations on negative cookie are dealt with in inline wrapper funcs.
     - Netfs must set file size on data object before accessing pages.

 (*) CacheFS:

     - Now based on a single metadata tree.
     - No discrete indexes.
     - Journalling much simplified: wandering tree algorithm used.
     - Full filesystem integrity _and_ data integrity.
     - More space available for cache.
     - Object entries consume one sector (512 bytes) each.
     - Object entries can hold up to ~400 bytes of key and aux data.
     - Objects can hold pages with index values up to 2^32 or 2^64.
     - Netfs aux object information updated on disk in passing on write page.

See the patched documentation for more specific information.

What remains to be done / future developments:

 (*) FS-Cache/general:

     - Userspace control operations (pioctl?):
       - Object pinning.
       - Object space reservation.
       - Object retirement.
     - Allocate page(s) with no read.
     - Write multiple pages.
     - Use kobjects for cache tags so they can be accessed through sysfs.
     - Lots of testing.

 (*) CacheFS:

     - Autowander timer (currently only wanders on allocator underrun, sync or
       unmount).
     - Tree scanner:
       - Object culling to make space.
       - Orphaned object deletion.
     - Issuing ENOBUFS if no space can be made available.
     - Use CacheFS mount as filesystem, perhaps for /tmp or ccache?

 (*) CacheFiles facility (caching through files on already mounted filesystem).


The patches can be downloaded from here:

 (*) http://people.redhat.com/~dhowells/cachefs/signcmp-fix-2612mm1-10.diff

	Fix a signed/unsigned comparison to prevent -Wsign-compare from
	complaining all over the place.

 (*) http://people.redhat.com/~dhowells/cachefs/fscache-bits-2612mm1-10.diff

	Some changes to the core code to enable the new stuff.

 (*) http://people.redhat.com/~dhowells/cachefs/fscache-docs-2612mm1-10.diff

	Updated documentation.

 (*) http://people.redhat.com/~dhowells/cachefs/fscache-core-2612mm1-10.diff

	FS-Cache changes.

 (*) http://people.redhat.com/~dhowells/cachefs/fscache-cachefs-2612mm1-10.diff

	CacheFS changes.

 (*) http://people.redhat.com/~dhowells/cachefs/fscache-afs-2612mm1-10.diff

	Cached AFS changes.

 (*) http://people.redhat.com/~dhowells/cachefs/fscache-nfs-2612mm1-10.diff

	Cached NFS changes.

These were made against 2.6.12-mm1.

And there's a debugging utility as well:

	http://people.redhat.com/~dhowells/cachefs/dump-cachefs.c

This can be used to examine a block device containing a cache.

David