[Linux-cachefs] Issues

Wed May 20 13:48:38 UTC 2009

What am I doing wrong?

Here is my setup.

One Centos 5 64 bit sharing out a nfs mount point, aka server.

One Centos 5 64 bit nfs client with the following software, aka client.
cachefilesd-0.9 installed from tarball
linux nfs-utils 1.1.5
Kernel 2.6.30-rc6

How I test.
I start out with a completely clean /var/fscache on the client, / is 
ext3 with mount options rw,user_xattr.  The nfs export from the server 
is mounted on the client as /nfs (nfs:/nfs on /nfs type nfs 
(rw,fsc,addr=10.1.134.71)).  The server has the 2.6.29 kernel tree 
untar'ed on it.   On the client I start a copying over the kernel tree 
from the server to my home dir on the client server using cp -a.  And 
cat /proc/fs/fscache/stats shows.

FS-Cache statistics
Cookies: idx=2 dat=133335 spc=0
Objects: alc=133338 nal=0 avl=133338 ded=116219
ChkAux : non=0 ok=0 upd=0 obs=0
Pages  : mrk=460795 unc=410000
Acquire: n=133337 nul=0 noc=0 ok=133337 nbf=0 oom=0
Lookups: n=133338 neg=133338 pos=0 crt=133338
Updates: n=0 nul=0 run=0
Relinqs: n=116219 nul=0 wcr=0
AttrChg: n=0 ok=0 nbf=0 oom=0 run=0
Allocs : n=0 ok=0 wt=0 nbf=0
Allocs : ops=0 owt=0
Retrvls: n=161495 ok=0 wt=19245 nod=161495 nbf=0 int=0 oom=0
Retrvls: ops=161495 owt=1553
Stores : n=460765 ok=460765 agn=0 nbf=0 oom=0
Stores : ops=166227 run=626992
Ops    : pend=1553 run=326169 enq=626992
Ops    : dfr=0 rel=327722 gc=0

I then remove the copy on the client server from my home dir and run 
again and the output of /proc/fs/fscache/stats shows.  This time cache 
is populated.

FS-Cache statistics
Cookies: idx=2 dat=266670 spc=0
Objects: alc=133339 nal=0 avl=133339 ded=133336
ChkAux : non=0 ok=1 upd=0 obs=0
Pages  : mrk=460795 unc=460795
Acquire: n=266672 nul=0 noc=133334 ok=133338 nbf=133334 oom=0
Lookups: n=133339 neg=133338 pos=1 crt=133338
Updates: n=0 nul=0 run=0
Relinqs: n=133336 nul=0 wcr=0
AttrChg: n=0 ok=0 nbf=0 oom=0 run=0
Allocs : n=0 ok=0 wt=0 nbf=0
Allocs : ops=0 owt=0
Retrvls: n=161496 ok=1 wt=19246 nod=161495 nbf=0 int=0 oom=0
Retrvls: ops=161496 owt=1553
Stores : n=460765 ok=460765 agn=0 nbf=0 oom=0
Stores : ops=166227 run=626992
Ops    : pend=1553 run=326170 enq=626993
Ops    : dfr=0 rel=327723 gc=0

The only real change I see between the two is the Acquire line, noc and 
nbf count rise.

Kernel config is
grep -i fscache .config
CONFIG_FSCACHE=m
CONFIG_FSCACHE_STATS=y
# CONFIG_FSCACHE_HISTOGRAM is not set
# CONFIG_FSCACHE_DEBUG is not set
CONFIG_NFS_FSCACHE=y

I have also repeated this test over with 2.6.30-rc{1,2,3,4,5}, the only 
kernel that seems to work correctly is 2.6.29-rc3 pulled from  
git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-2.6-nfs-fscache.git 
a while back.  It is also setup with the same kernel config.

Here is the output from /proc/fs/fscache/stats from the old kernel I 
built from the git repo, same test from a clean /var/fscache

FS-Cache statistics
Cookies: idx=2 dat=133335 spc=0
Objects: alc=133338 nal=0 avl=133338 ded=115184
ChkAux : non=0 ok=0 upd=0 obs=0
Pages  : mrk=460795 unc=411949
Acquire: n=133337 nul=0 noc=0 ok=133337 nbf=0 oom=0
Lookups: n=133338 neg=133338 pos=0 crt=133338
Updates: n=0 nul=0 run=0
Relinqs: n=115184 nul=0 wcr=0
AttrChg: n=0 ok=0 nbf=0 oom=0 run=0
Allocs : n=0 ok=0 wt=0 nbf=0
Allocs : ops=0 owt=0
Retrvls: n=161495 ok=0 wt=35757 nod=161495 nbf=0 int=0 oom=0
Retrvls: ops=161495 owt=30335
Stores : n=460765 ok=460765 agn=0 nbf=0 oom=0
Stores : ops=166940 run=627705
Ops    : pend=30335 run=298100 enq=627705
Ops    : dfr=0 rel=328435 gc=0

I then remove the copy on the client server from my home dir and run 
again and the output of /proc/fs/fscache/stats shows.  This time cache 
is populated.

FS-Cache statistics
Cookies: idx=2 dat=266670 spc=0
Objects: alc=266673 nal=0 avl=266673 ded=248592
ChkAux : non=0 ok=133335 upd=0 obs=0
Pages  : mrk=921590 unc=870190
Acquire: n=266672 nul=0 noc=0 ok=266672 nbf=0 oom=0
Lookups: n=266673 neg=133338 pos=133335 crt=133338
Updates: n=0 nul=0 run=0
Relinqs: n=248592 nul=0 wcr=0
AttrChg: n=0 ok=0 nbf=0 oom=0 run=0
Allocs : n=0 ok=0 wt=0 nbf=0
Allocs : ops=0 owt=0
Retrvls: n=322990 ok=161463 wt=155920 nod=161527 nbf=0 int=0 oom=0
Retrvls: ops=322990 owt=30335
Stores : n=460768 ok=460768 agn=0 nbf=0 oom=0
Stores : ops=166941 run=627709
Ops    : pend=30335 run=459596 enq=1099839
Ops    : dfr=0 rel=489931 gc=0

Any ideas or tips on getting this working correctly with the 2.6.30-rc* 
kernels?

Thanks
Edward