[dm-devel] Cache writethrough mode skips writes to source

[Jens-Wolfhard Schicke-Uffmann]

Hi,

while debugging an I/O load issue at a client, I came across cases of system
reboots (which apparently included an unclear device shutdown), where a
dm-cache was marked all-dirty, as described e.g. in admin-guide/device-mapper/cache.rst
> The 'dirty' state for a cache block changes far too frequently for us to keep
> updating it on the fly. So we treat it as a hint. In normal operation it will
> be written when the dm device is suspended. If the system crashes all cache
> blocks will be assumed dirty when restarted.

As the caches were in front of a RAID1 they were configured "writethrough" in order
not to reduce data redundancy. As per documentation:
> write through caching that prohibits cache block content from being
> different from origin block content.

While trying to understand caching behavior in more detail, I noticed that in
dm-cache-target.c, map_bio this guarantee seems to be violated

if (passthrough_mode(cache)) {
	if (bio_data_dir(bio) == WRITE) {
		bio_drop_shared_lock(cache, bio);
		atomic_inc(&cache->stats.demotion);
		invalidate_start(cache, cblock, block, bio);
	} else
		remap_to_origin_clear_discard(cache, bio, block);
} else {
	if (bio_data_dir(bio) == WRITE && writethrough_mode(cache) &&
			!is_dirty(cache, cblock)) {
                          // ^----- !!!BUG!!!
		remap_to_origin_and_cache(cache, bio, block, cblock);
		accounted_begin(cache, bio);
	} else
		remap_to_cache_dirty(cache, bio, block, cblock);
}

... in case a writethrough cache ever gets dirty cblocks, which is to say
after an unclean shutdown.

And indeed, a quick comparison of device contents showed a lot of differences
between the cached block device and the source RAID1.


It seems to me that the is_dirty condition can simply be dropped, but the
comment above remap_to_origin_and_cache()
> When running in writethrough mode we need to send writes to clean blocks
suggests there is a reason why it exists.

- Jens