[Linux-cachefs] Fscache is running but take no caching effection
WeiFeng
liuwf at sanlogic.com
Wed Dec 19 03:12:09 UTC 2012
Hello, everyone
In the last two days I installed cachefiles and make some tests on Centos 6.3
with kernel 3.5.3 servers. All tests were done for a pair servers and
for single server, the cache dir are made on root files system with ext4 and
on ramdisk with ext4, but no caching action were observed, there are still high
network access between two servers for large files, and no FS caching except
memory caching for small files(4GB RAM equiped for test servers). Selinux
and iptables are all disabled before tests.
Followings are records from tests on single server, test records for files size>4GB
are not post here, but had same results in no FS caching actions. Can anyone tell
me if I'v missed something or done wrong things?
--------------------------------------------------------
my cachefilesd conf file:
dir /fscache
tag mycache
brun 10%
bcull 7%
bstop 3%
frun 10%
fcull 7%
fstop 3%
#secctx system_u:system_r:cachefiles_kernel_t:s0
mkdir /fscache
[root at sanhost ~]# service cachefilesd start
Starting cachefilesd: [ OK ]
[root at sanhost ~]# service cachefilesd status
cachefilesd (pid 3894) is running....
[root at sanhost ~]# ls -l /fscache/
total 8
drwx------ 2 root root 4096 Dec 19 09:05 cache
drwx------ 2 root root 4096 Dec 19 09:05 graveyard
[root at sanhost ~]# dmesg
...
FS-Cache: Withdrawing cache "mycache"
FS-Cache: Cache "mycache" added (type cachefiles)
CacheFiles: File cache on sda6 registered
[root at sanhost ~]# mount -t nfs4 -o fsc,rsize=32768,wsize=32768,async 127.0.0.1:/nfs_share y
[root at sanhost ~]# dd if=/dev/zero bs=1G count=4 of=y/img01
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 11.024 s, 97.4 MB/s
[root at sanhost ~]# mount y
[root at sanhost ~]# mount -t nfs4 -o fsc,rsize=32768,wsize=32768,async 127.0.0.1:/nfs_share y
[root at sanhost ~]# time cat y/img01 >/dev/null
real 0m1.374s
user 0m0.007s
sys 0m0.582s
[root at sanhost ~]# time cat y/img01 >/dev/null
real 0m0.258s
user 0m0.005s
sys 0m0.251s
[root at sanhost ~]#
[root at sanhost ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda6 15748508 8916284 6032224 60% /
tmpfs 2010308 76 2010232 1% /dev/shm
/dev/sda2 186671 149497 27535 85% /boot
127.0.0.1:/nfs_share 15748512 8916288 6032224 60% /root/y
[root at sanhost ~]# cat /proc/fs/fscache/stats
FS-Cache statistics
Cookies: idx=12 dat=0 spc=0
Objects: alc=0 nal=0 avl=0 ded=2
ChkAux : non=0 ok=0 upd=0 obs=0
Pages : mrk=0 unc=0
Acquire: n=12 nul=0 noc=0 ok=12 nbf=0 oom=0
Lookups: n=0 neg=0 pos=0 crt=0 tmo=0
Updates: n=0 nul=0 run=0
Relinqs: n=10 nul=0 wcr=0 rtr=0
AttrChg: n=0 ok=0 nbf=0 oom=0 run=0
Allocs : n=0 ok=0 wt=0 nbf=0 int=0
Allocs : ops=0 owt=0 abt=0
Retrvls: n=0 ok=0 wt=0 nod=0 nbf=0 int=0 oom=0
Retrvls: ops=0 owt=0 abt=0
Stores : n=0 ok=0 agn=0 nbf=0 oom=0
Stores : ops=0 run=0 pgs=0 rxd=0 olm=0
VmScan : nos=0 gon=0 bsy=0 can=0
Ops : pend=0 run=0 enq=0 can=0 rej=0
Ops : dfr=0 rel=0 gc=0
CacheOp: alo=0 luo=0 luc=0 gro=0
CacheOp: upo=0 dro=0 pto=0 atc=0 syn=0
CacheOp: rap=0 ras=0 alp=0 als=0 wrp=0 ucp=0 dsp=0
[root at sanhost ~]# cat /var/log/messages |tail
Dec 19 09:10:55 sanhost cachefilesd[3787]: Daemon Terminated
Dec 19 09:10:55 sanhost kernel: CacheFiles: File cache on sda6 unregistering
Dec 19 09:10:55 sanhost kernel: FS-Cache: Withdrawing cache "mycache"
Dec 19 09:12:02 sanhost cachefilesd[3893]: About to bind cache
Dec 19 09:12:02 sanhost cachefilesd[3893]: Bound cache
Dec 19 09:12:02 sanhost kernel: FS-Cache: Cache "mycache" added (type cachefiles)
Dec 19 09:12:02 sanhost kernel: CacheFiles: File cache on sda6 registered
Dec 19 09:12:02 sanhost cachefilesd[3894]: Daemon Started
Dec 19 09:17:47 sanhost gnome-keyring-daemon[2778]: removing removable location: /root/y
Dec 19 09:17:47 sanhost gnome-keyring-daemon[2778]: no volume registered at: /root/y
[root at sanhost ~]# ls -l /proc/sys/fs/nfs/
total 0
-rw-r--r-- 1 root root 0 Dec 19 09:28 idmap_cache_timeout
-rw-r--r-- 1 root root 0 Dec 19 09:28 nfs_callback_tcpport
-rw-r--r-- 1 root root 0 Dec 19 09:28 nfs_congestion_kb
-rw-r--r-- 1 root root 0 Dec 19 09:28 nfs_mountpoint_timeout
-rw-r--r-- 1 root root 0 Dec 19 09:28 nlm_grace_period
-rw-r--r-- 1 root root 0 Dec 19 09:28 nlm_tcpport
-rw-r--r-- 1 root root 0 Dec 19 09:28 nlm_timeout
-rw-r--r-- 1 root root 0 Dec 19 09:28 nlm_udpport
-rw-r--r-- 1 root root 0 Dec 19 09:28 nsm_local_state
-rw-r--r-- 1 root root 0 Dec 19 09:28 nsm_use_hostnames
[root at sanhost ~]# cat /proc/fs/nfsfs/servers
NV SERVER PORT USE HOSTNAME
v4 7f000001 801 1 127.0.0.1
[root at sanhost ~]# cat /proc/fs/nfsfs/volumes
NV SERVER PORT DEV FSID FSC
v4 7f000001 801 0:32 1c53587af4c14aff yes
[root at sanhost ~]# du -sh /fscache/
12K /fscache/
#my kernel option:
...
CONFIG_NFS_FSCACHE=y
...
CONFIG_FSCACHE=m
CONFIG_FSCACHE_STATS=y
# CONFIG_FSCACHE_HISTOGRAM is not set
# CONFIG_FSCACHE_DEBUG is not set
# CONFIG_FSCACHE_OBJECT_LIST is not set
CONFIG_CACHEFILES=m
# CONFIG_CACHEFILES_DEBUG is not set
# CONFIG_CACHEFILES_HISTOGRAM is not set
#cachefiles dir on a ram disk
[root at sanhost ~]# dd if=/dev/zero bs=1G count=1 of=/dev/shm/ramdisk
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 1.16788 s, 919 MB/s
[root at sanhost ~]# mkfs.ext4 /dev/shm/ramdisk
mke2fs 1.41.12 (17-May-2010)
/dev/shm/ramdisk is not a block special device.
Proceed anyway? (y,n) y
...
[root at sanhost ~]# mount /dev/shm/ramdisk /fscache -o user_xattr,loop
[root at sanhost ~]# service cachefilesd start
Starting cachefilesd: [ OK ]
[root at sanhost ~]# mount -t nfs4 -o fsc,rsize=32768,wsize=32768,async 127.0.0.1:/nfs_share y
[root at sanhost ~]# dd if=/dev/zero bs=128M count=1 of=y/img02
1+0 records in
1+0 records out
134217728 bytes (134 MB) copied, 1.64453 s, 81.6 MB/s
[root at sanhost ~]# time cat y/img02 >/dev/null
real 0m0.054s
user 0m0.000s
sys 0m0.032s
[root at sanhost ~]# time cat y/img02 >/dev/null
real 0m0.036s
user 0m0.000s
sys 0m0.035s
[root at sanhost ~]# du -sh /fscache/
28K /fscache/
[root at sanhost ~]# ls -l /fscache/
total 24
drwx------ 2 root root 4096 Dec 19 09:41 cache
drwx------ 2 root root 4096 Dec 19 09:41 graveyard
drwx------ 2 root root 16384 Dec 19 09:40 lost+found
[root at sanhost ~]# service cachefilesd status
cachefilesd (pid 4519) is running...
[root at sanhost ~]# ls -l /proc/sys/fs/nfs/
total 0
-rw-r--r-- 1 root root 0 Dec 19 09:28 idmap_cache_timeout
-rw-r--r-- 1 root root 0 Dec 19 09:28 nfs_callback_tcpport
-rw-r--r-- 1 root root 0 Dec 19 09:28 nfs_congestion_kb
-rw-r--r-- 1 root root 0 Dec 19 09:28 nfs_mountpoint_timeout
-rw-r--r-- 1 root root 0 Dec 19 09:28 nlm_grace_period
-rw-r--r-- 1 root root 0 Dec 19 09:28 nlm_tcpport
-rw-r--r-- 1 root root 0 Dec 19 09:28 nlm_timeout
-rw-r--r-- 1 root root 0 Dec 19 09:28 nlm_udpport
-rw-r--r-- 1 root root 0 Dec 19 09:28 nsm_local_state
-rw-r--r-- 1 root root 0 Dec 19 09:28 nsm_use_hostnames
[root at sanhost ~]# cat /proc/fs/nfsfs/volumes
NV SERVER PORT DEV FSID FSC
v4 7f000001 801 0:32 1c53587af4c14aff yes
[root at sanhost ~]# cat /proc/fs/nfsfs/servers
NV SERVER PORT USE HOSTNAME
v4 7f000001 801 1 127.0.0.1
[root at sanhost ~]# cat /proc/fs/fscache/stats
FS-Cache statistics
Cookies: idx=15 dat=0 spc=0
Objects: alc=0 nal=0 avl=0 ded=3
ChkAux : non=0 ok=0 upd=0 obs=0
Pages : mrk=0 unc=0
Acquire: n=15 nul=0 noc=0 ok=15 nbf=0 oom=0
Lookups: n=0 neg=0 pos=0 crt=0 tmo=0
Updates: n=0 nul=0 run=0
Relinqs: n=13 nul=0 wcr=0 rtr=0
AttrChg: n=0 ok=0 nbf=0 oom=0 run=0
Allocs : n=0 ok=0 wt=0 nbf=0 int=0
Allocs : ops=0 owt=0 abt=0
Retrvls: n=0 ok=0 wt=0 nod=0 nbf=0 int=0 oom=0
Retrvls: ops=0 owt=0 abt=0
Stores : n=0 ok=0 agn=0 nbf=0 oom=0
Stores : ops=0 run=0 pgs=0 rxd=0 olm=0
VmScan : nos=0 gon=0 bsy=0 can=0
Ops : pend=0 run=0 enq=0 can=0 rej=0
Ops : dfr=0 rel=0 gc=0
CacheOp: alo=0 luo=0 luc=0 gro=0
CacheOp: upo=0 dro=0 pto=0 atc=0 syn=0
CacheOp: rap=0 ras=0 alp=0 als=0 wrp=0 ucp=0 dsp=0
--------------------------------------------------------
thanks
WeiFeng
More information about the Linux-cachefs
mailing list