[dm-devel] dm-cache: can the same cache be used with multiple origin devices?
Mike Snitzer
snitzer at redhat.com
Mon Jul 22 14:35:37 UTC 2013
On Mon, Jul 15 2013 at 3:59pm -0400,
Mike Snitzer <snitzer at redhat.com> wrote:
> On Mon, Jul 15 2013 at 3:01pm -0400,
> Mears, Morgan <Morgan.Mears at netapp.com> wrote:
>
> > Hi,
> >
> > In reference to dm-cache: can the same cache and metadata devices be
> > used with multiple origin devices? This can be configured, and we've
> > done some tests that appear to show that it works - we're looking for
> > confirmation (or otherwise).
>
> It is _not_ supported.
>
> > Here's an example test setup to clarify -- ssd_metadata and ssd_blocks
> > are being used to cache sdc and sdd. In testing, different patterns
> > were written to areas of sdc_cached and sdd_cached; afterwards, the
> > contents of sdc and sdd were as expected.
> >
> > dmsetup create sdc_cached --table '0 4194304 cache /dev/mapper/ssd_metadata /dev/mapper/ssd_blocks /dev/sdc 512 1 writethrough default 0'
> > dmsetup create sdd_cached --table '0 4194304 cache /dev/mapper/ssd_metadata /dev/mapper/ssd_blocks /dev/sdd 512 1 writethrough default 0'
>
> Interesting.
>
> The current cache target obviously fails to detect that the metadata or
> data devices are already in use. But that doesn't mean it is safe to
> utilize the cache in this mode (I'll have a think about where the code
> will break down). But the cache is managed/written in a manner that
> only assumes a single backing origin for each cache.
The dm-cache code is inherently unsafe to use in the above configuration
because it results in disjoint tasks accessing the same metadata
device. This is inherently racey because cmd->root_lock (metadata) and
cache->lock (cache device) are completely independent.
So things like tearing down one cache device (sdc_cached) while
issuing discards to the other (sdd_cached) would be one example of
competing tasks stepping all over one another. More fundamental actions
like allocating a new cache block is racey. Etc.
There are many other potential problems -- you apparently have just been
"lucky" not to hit them yet. These races would likely become much more
transparent if you created 10 sdX_cached devices that share the same
metadata device and you then data integrity sensitive workloads against
each sdX_cached device (like repeat linux.git checkouts or even netapp's
dt).
Mike
More information about the dm-devel
mailing list