[dm-devel] dm-cache with zero hit rate

[Steinar H. Gunderson]

On Wed, Jul 31, 2013 at 04:25:22PM -0400, Mike Snitzer wrote:
>> After a fair amount of fighting with udev and initramfs-tools, plus upping
>> the block size to 2048 since 512 complained about not enough RAM (on a 24GB
>> machine!), this seems to boot up and work, but I seem to get absolutely zero
>> cache hits.
> Kudos to you for hacking that to work.

It's actually not that hard; once things stabilize and I know this is
actually the solution I want to have, I plan to sit down and document it, and
it's not going to be a long document.

> Please see the recent "dm-cache warming thread":
> https://www.redhat.com/archives/dm-devel/2013-July/msg00133.html
> 
> That isn't to say we cannot take steps to be more aggressive; but we'll
> need more context for what you're doing.
> 
> A normal system boot is likely predominantly read once, and/or (as Joe
> pointed out) the page cache could be masking subsequent reads.
> 
> If you're doing write heavy workloads are they being elided by the
> sequential_threshold?
> 
> Try a git checkout, and switch branches a few times (e.g. checkout v3.1,
> then v3.8, then v3.2, then v3.9, then v3.1, etc).

OK, so context: This is my personal do-everything server, so it does all
kinds of things; web serving (both pages, scripts and mirrors), large files,
database work, email (local, not IMAP), backup through rsync, etc.. Although
it has 24GB of RAM, the disk set is 8TB (and the cache disks ~450GB), so one
would think it found _something_ relevant to cache. In particular, my
Maildirs would seem like a prime candidate :-) There's I/O going on all the
time, and I'm a bit surprised that not even any writes are cached. 

I did the git checkout as you asked, 10–15 times with different version
numbers, and although the git command seemed to create I/O (ie., the page
cache wasn't gobbling it all up, although it was mostly write I/O), but it
still didn't put anything in the cache.

I see the thread you're referring to says “too low”, but I really have zero,
even after almost 40 hours of normal operation:

  cache: 0 23440891904 cache 913/8192 0 6888144 0 1653677 0 0 0 0 0 2 migration_threshold 2048 4 random_threshold 8 sequential_threshold 512

Could the large block size be related to this?

/* Steinar */
-- 
Homepage: http://www.sesse.net/