[dm-devel] [PATCH blktests] dm: add a regression test

[Shin'ichiro Kawasaki]

On Apr 26, 2023 / 08:42, Chaitanya Kulkarni wrote:
> On 4/25/23 10:24, Mike Snitzer wrote:
> > On Tue, Apr 25 2023 at  8:15P -0400,
> > Shin'ichiro Kawasaki <shinichiro at fastmail.com> wrote:
> >
> >> On Apr 25, 2023 / 16:22, Yu Kuai wrote:
> >>> Hi,
> >>>
> >>> 在 2023/01/12 9:05, Shinichiro Kawasaki 写道:
> >>>> Hello Yu, thanks for the patch. I think it is good to have this test case to
> >>>> avoid repeating the same regression. Please find my comments in line.
> >>>>
> >>>> CC+: Mike, dm-devel,
> >>>>
> >>>> Mike, could you take a look in this new test case? It adds "dm" test group to
> >>>> blktests. If you have thoughts on how to add device-mapper related test cases
> >>>> to blktests, please share (Or we may be able to discuss later at LSF 2023).
> >>> Can we add "dm" test group to blktests? I'll send a new version if we
> >>> can.
> >> I suggest to wait for LSF discussion in May, which could be a good chance to
> >> hear opinions of dm experts. I think your new test case is valuable, so IMO it
> >> should be added to the new "dm" group, or at least to the existing "block"
> >> group. Let's decide the target group after LSF.
> >>
> > It's obviously fine to add a new "dm" test group to blktests.

Mike, thanks for the positive comment. Now I know that there are various
testsuites related to dm as you noted below. I think the new dm test group in
blktests can have different coverage from other dm related testsuites: tests to
confirm fixes for failures related to block layer and device-mapper. The new
test Yu Kuai suggests will fit into it.

Yu, could you post the new version of your patch? Let's add the dm group and the
first test case.

> >
> > But just so others are aware: more elaborate dm testing is currently
> > spread across multiple testsuites (e.g. lvm2, cryptsetup, mptest,
> > device-mapper-test-suite, etc).

Thanks. Good to know that dm test has wide varieties. I found the testsuites on
the net. Will try to look into them.

[1] lvm2: https://github.com/lvmteam/lvm2/tree/master/test
[2] cryptsetup: https://gitlab.com/cryptsetup/cryptsetup/-/tree/main/tests
[3] mptest: https://github.com/snitm/mptest
[4] device-mapper-test-suite: https://github.com/jthornber/device-mapper-test-suite

> >
> > There is new effort to port device-mapper-test-suite tests (which use
> > ruby) to a new python harness currently named "dmtest-python", Joe
> > Thornber is leading this effort (with the assistance of
> > ChatGPT.. apparently it has been wonderful in helping Joe glue python
> > code together to accomplish anything he's needed):
> > https://github.com/jthornber/dmtest-python
> >
> > (we've discussed renaming "dmtest-python" to "dmtests")
> >
> > I've also discussed with Joe the plan to wrap the other disparate
> > testsuites with DM coverage in terms of the new dmtest-python.

It sounds a valuable action to gather the various testsuites so that the dm
maintainer or dm developers can run them all easily.

I also think that the list of testsuites will help developers to tell what kind
of test goes to which testsuite. When an developer adds a test case related to
dm, the developer needs to choose which testsuite to add it. As an example, I
wonder about dm-zoned. In case we would add test cases to exercise dm-zoned, is
there any good testsuite to add them? (Can we set up dm-zoned with dmtest-
python?)

> > blktests can be made to be one of the testsuites we add support for
> > (so that all blktests run on various types of DM targets).
> >
> > Really all we need is a means to:
> > 1) list all tests in a testsuite
> > 2) initiate running each test individually

IIUC, the dmtest-python takes the role to prepare the various DM targets for
test, and each testsuite runs test cases on the targets, right? From blktests
point of view, the "block" group of blktests can be run with the various DM
targets. Also the "zbd" group of blktests can be run with DM targets with
DM_TARGET_ZONED_HM feature. Other test groups such as "nvme" or "scsi" won't
run with DM targets. I think it is not difficult for dmtest-python to prepare
blktests config files to specify DM targets and the block/zbd test group. If
blktests side change is desired, we can discuss.

> >
> > Mike
> 
> Thanks Mike for the detailed information, we did talk about DM testcases
> in last LSFMM, this is really important piece of blktest that is missing
> and need to be discussed this year's LSFMM so we can integrate above
> work in blktests as right now we are not able to establish complete
> stability due to lack of of the dm tests as we are doing it for block
> layer code or nvme for example.
> 
> -ck
> 
>