[dm-devel] multipath target and non-request-stackable devices
Mike Snitzer
snitzer at redhat.com
Wed Oct 31 22:14:03 UTC 2018
On Wed, Oct 31 2018 at 5:32am -0400,
Drew Hastings <dhastings at crucialwebhost.com> wrote:
> Firstly, thanks for the hard work you guys are doing on the dm drivers.
> I'm curious to know if I correctly understand the limitations of the
> multipath target. I'm using kernel 4.19.0-rc5
> When attempting to create a device from two NVMEs connected over nvme_rdma
> / nvmf, I get the following message:
>
> [43012.002418] device-mapper: table: table load rejected: including
> non-request-stackable devices
> [43012.004067] device-mapper: table: unable to determine table type
> Here's an example request that will fail:
> dmsetup create test_path --table "0 1562824368 multipath 0 0 2 1
> round-robin 0 1 1 /dev/nvme1n1 1 round-robin 0 1 1 /dev/nvme2n1 1"
dm-multipath works fine ontop of nvme. Both blktests and mptest have
test coverage for dm-multipath on nvme:
https://github.com/osandov/blktests
https://github.com/snitm/mptest
That said, there are 2 different modes that can be used when creating
the dm-multipath device, The default is request-based ("mq") and there
is also bio-based ("bio"), e.g.:
queue_mode mq
or
queue_mode bio
It should be noted that in 4.20 request-based DM _only_ supports blk-mq
("queue_mode mq"). Using "rq" just means "mq" (and vice-versa) as of
4.20.
The error you've encountered shouldn't occur given that nvme devices use
blk-mq for their request_queue. It could be that 4.19-rc5 had a bug
that was fixed but nothing springs to mind.
But I do recall hitting such an error during development and testing.
And dm_table_determine_type() still isn't as "clean" as I'd like...
> This will work:
> dmsetup create test_path --table "0 1562824368 multipath 0 0 2 1
> round-robin 0 1 1 /dev/sdc 1 round-robin 0 1 1 /dev/sdb 1"
> After looking at dm-table.c, it seems like this may be an issue of these
> underlying devices not being compatible with the multipath driver. I had
> noticed this same error when trying "non-physical" devices in other tests,
> such as trying to multipath other virtual devices.
> Is there anything that can be done to do the (relatively "basic")
> multipath that detects IO errors and switches to the other device? It
> seems like fundamentally multipath's error detection works in the same way
> raid1 implements IO error detection of the underlying devices, so why is
> it that multipath seems to have this limitation but raid1 does not?
request-based DM can only be stacked on request-based devices. MD raid1
is bio-based, bio-based can stack on anything.
But again: nvme uses blk-mq.
> Is md-multipath.c still in use? It seems like it might not have this same
> restriction on the underlying device.
I don't know of anyone who uses MD multipath.
Mike
More information about the dm-devel
mailing list