[Linux-cachefs] Fwd: Fscache is running but take no caching effection

Mon Jan 14 18:30:09 UTC 2013

Forwarding on to the 

-------- Original Message --------
Subject: Fscache is running but take no caching effection
Date: Mon, 14 Jan 2013 16:57:12 +0800
From: 刘伟丰 <liuwf at sanlogic.com>
To: linux-nfs <linux-nfs at vger.kernel.org>

Hi all

I installed cachefiles and made some tests for NFS on Centos 6.3 with kernel 3.5.3 servers. All tests were done both on a pair servers and on single server, the cache dir were made on root files system with ext4 , but no caching action were observed, there were still high network transmission between two servers for large files, and no fs caching except memory caching for small files(4GB RAM equiped for test servers). Selinux and Iptables are all disabled before tests in the case.

BTW, I got fs caching work well by using smb(cifs) + cachefiles with the same configures following.

Followings are records from tests on single server, test records for files size>4GB are not post here, but had the same results without any fs caching actions. Could anyone tell me if I'v missed something?

--------------------------------------------------------
cachefilesd conf file:

dir /fscache
tag mycache
brun 10%
bcull 7%
bstop 3%
frun 10%
fcull 7%
fstop 3%
#secctx system_u:system_r:cachefiles_kernel_t:s0

mkdir /fscache
[root at sanhost ~]# service cachefilesd start
Starting cachefilesd:                                      [  OK  ]
[root at sanhost ~]# service cachefilesd status
cachefilesd (pid  3894) is running....
[root at sanhost ~]# ls -l /fscache/
total 8
drwx------ 2 root root 4096 Dec 19 09:05 cache
drwx------ 2 root root 4096 Dec 19 09:05 graveyard
[root at sanhost ~]# dmesg
...
FS-Cache: Withdrawing cache "mycache"
FS-Cache: Cache "mycache" added (type cachefiles)
CacheFiles: File cache on sda6 registered

[root at sanhost ~]# mount -t nfs4 -o fsc,rsize=32768,wsize=32768,async 127.0.0.1:/nfs_share y
[root at sanhost ~]# dd if=/dev/zero bs=1G count=1 of=y/img01
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 11.024 s, 97.4 MB/s

[root at sanhost ~]# umount y
[root at sanhost ~]# mount -t nfs4 -o fsc,rsize=32768,wsize=32768,async 127.0.0.1:/nfs_share y
[root at sanhost ~]# time cat y/img01 >/dev/null
real    0m10.374s
user    0m0.007s
sys     0m0.582s
[root at sanhost ~]# time cat y/img01 >/dev/null
real    0m0.258s
user    0m0.005s
sys     0m0.251s
[root at sanhost ~]#
[root at sanhost ~]# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda6             15748508   8916284   6032224  60% /
tmpfs                  2010308        76   2010232   1% /dev/shm
/dev/sda2               186671    149497     27535  85% /boot
127.0.0.1:/nfs_share  15748512   8916288   6032224  60% /root/y
[root at sanhost ~]# cat /proc/fs/fscache/stats
FS-Cache statistics
Cookies: idx=12 dat=0 spc=0
Objects: alc=0 nal=0 avl=0 ded=2
ChkAux : non=0 ok=0 upd=0 obs=0
Pages  : mrk=0 unc=0
Acquire: n=12 nul=0 noc=0 ok=12 nbf=0 oom=0
Lookups: n=0 neg=0 pos=0 crt=0 tmo=0
Updates: n=0 nul=0 run=0
Relinqs: n=10 nul=0 wcr=0 rtr=0
AttrChg: n=0 ok=0 nbf=0 oom=0 run=0
Allocs : n=0 ok=0 wt=0 nbf=0 int=0
Allocs : ops=0 owt=0 abt=0
Retrvls: n=0 ok=0 wt=0 nod=0 nbf=0 int=0 oom=0
Retrvls: ops=0 owt=0 abt=0
Stores : n=0 ok=0 agn=0 nbf=0 oom=0
Stores : ops=0 run=0 pgs=0 rxd=0 olm=0
VmScan : nos=0 gon=0 bsy=0 can=0
Ops    : pend=0 run=0 enq=0 can=0 rej=0
Ops    : dfr=0 rel=0 gc=0
CacheOp: alo=0 luo=0 luc=0 gro=0
CacheOp: upo=0 dro=0 pto=0 atc=0 syn=0
CacheOp: rap=0 ras=0 alp=0 als=0 wrp=0 ucp=0 dsp=0

[root at sanhost ~]# cat /var/log/messages |tail
Dec 19 09:10:55 sanhost cachefilesd[3787]: Daemon Terminated
Dec 19 09:10:55 sanhost kernel: CacheFiles: File cache on sda6 unregistering
Dec 19 09:10:55 sanhost kernel: FS-Cache: Withdrawing cache "mycache"
Dec 19 09:12:02 sanhost cachefilesd[3893]: About to bind cache
Dec 19 09:12:02 sanhost cachefilesd[3893]: Bound cache
Dec 19 09:12:02 sanhost kernel: FS-Cache: Cache "mycache" added (type cachefiles)
Dec 19 09:12:02 sanhost kernel: CacheFiles: File cache on sda6 registered
Dec 19 09:12:02 sanhost cachefilesd[3894]: Daemon Started
Dec 19 09:17:47 sanhost gnome-keyring-daemon[2778]: removing removable location: /root/y
Dec 19 09:17:47 sanhost gnome-keyring-daemon[2778]: no volume registered at: /root/y

[root at sanhost ~]# ls -l /proc/sys/fs/nfs/
total 0
-rw-r--r-- 1 root root 0 Dec 19 09:28 idmap_cache_timeout
-rw-r--r-- 1 root root 0 Dec 19 09:28 nfs_callback_tcpport
-rw-r--r-- 1 root root 0 Dec 19 09:28 nfs_congestion_kb
-rw-r--r-- 1 root root 0 Dec 19 09:28 nfs_mountpoint_timeout
-rw-r--r-- 1 root root 0 Dec 19 09:28 nlm_grace_period
-rw-r--r-- 1 root root 0 Dec 19 09:28 nlm_tcpport
-rw-r--r-- 1 root root 0 Dec 19 09:28 nlm_timeout
-rw-r--r-- 1 root root 0 Dec 19 09:28 nlm_udpport
-rw-r--r-- 1 root root 0 Dec 19 09:28 nsm_local_state
-rw-r--r-- 1 root root 0 Dec 19 09:28 nsm_use_hostnames

[root at sanhost ~]# cat /proc/fs/nfsfs/servers
NV SERVER   PORT USE HOSTNAME
v4 7f000001  801   1 127.0.0.1

[root at sanhost ~]# cat /proc/fs/nfsfs/volumes
NV SERVER   PORT DEV     FSID              FSC
v4 7f000001  801 0:32    1c53587af4c14aff  yes
[root at sanhost ~]# du -sh /fscache/
12K     /fscache/

#my kernel option:
...
CONFIG_NFS_FSCACHE=y
...
CONFIG_FSCACHE=m
CONFIG_FSCACHE_STATS=y
# CONFIG_FSCACHE_HISTOGRAM is not set
# CONFIG_FSCACHE_DEBUG is not set
# CONFIG_FSCACHE_OBJECT_LIST is not set
CONFIG_CACHEFILES=m
# CONFIG_CACHEFILES_DEBUG is not set
# CONFIG_CACHEFILES_HISTOGRAM is not set

thanks
WeiFengN‹§²æìr¸›yúèšØb²X¬¶Ç§vØ^–)Þº{.nÇ+‰·¥Š{±û"žØ^n‡r¡ö¦zËëh™¨èÚ&¢ø
®G«éh®(éšŽŠÝ¢j"ú¶m§ÿïêäz¹Þ–Šàþf£¢·hšˆ§~ˆmml==