[linux-lvm] lvm2-testsuite stability

[Zdenek Kabelac]

Dne 15. 06. 23 v 20:02 Scott Moser napsal(a):
> Hi,
> [sorry for duplicate post, re-sending from a subscribed address]
> 
> I'm looking to enable the lvm2 testsuite as an autopkgtest [1] to run
> in debian and ubuntu. I have a merge request up at [2].  The general
> idea is just to a.) package 'lvm2-testsuite' as an installable package
> b.) run the testsuite as part of the autopkgtest.
> 
> The version I'm testing on Ubuntu 22.04 is 2.03.16-3 from debian
> (rebuilt for 22.04). I'm running udev-vanilla  in a 2 cpu/4GB VM, and
> stopping/masking  the following services: dm-event lvm2-lvmpolld
> lvm2-monitor lvm2-lvmdbusd .
> 
> I'm seeing some failures when running the test.  Some seem expected
> due to size limitations, some seem to fail every time, and some see
> transient failures.
> 
> Here is the list of tests that I'm seeing fail and my initial
> categorization.  I've seen this across say half a dozen runs:
> 

Yep - some tests are failing

> expected-fail  api/dbustest.sh

We do have them even split to individual tests;

api/dbus_test_cache_lv_create.sh																							
api/dbus_test_copy_signature.sh																																			
api/dbus_test_external_event.sh																																			
api/dbus_test_log_file_option.sh																																			
api/dbus_test_wipefs.sh																																			
api/dbus_test_z_sigint.sh

these need to be fixed and resolved.

> expected-fail  shell/lvconvert-repair-thin.sh



> space-req      shell/lvcreate-large-raid.sh
> space-req      shell/lvcreate-thin-limits.sh
> expected-fail  shell/lvm-conf-error.sh
> expected-fail  shell/lvresize-full.sh
> timeout        shell/pvmove-abort-all.sh
> space-req      shell/pvmove-basic.sh
> expected-fail  shell/pvscan-autoactivation-polling.sh
> expected-fail  shell/snapshot-merge.sh
> space-req      shell/thin-large.sh
> racy           shell/writecache-cache-blocksize.sh

These are individual - we have some of those testing on some machines.
They may need some 'extra' care.

> 
> expected-fail fails most every time. timeout seems to work sometimes,
> space-req i think is just space requirement issue (i'll just skip
> those tests).
> 

I'd likely need to get access/see  to the logs of such machines
(or you would need to provide as some downloadable image of you Qemu machine 
installation)


> The full output from the test run can be seen at [3] in the
> testsuite-stdout.txt and testsuite-stderr.txt files.
> 
> Do others run this test-suite in automation and get reliable results ?
> 

We surely do run these tests on regular basis on VM - so those are usually 
slightly modified to avoid collisions with tests.
There is also no strict rule to not break some 'tests' - so occasionally some 
tests can be failing for a while if they are seen 'less important' over some 
other bugs...

We would need to think much harder if the test should be running with some 
daemons or autoactivation on the system that could see and could interact with 
our devices generated during the test run (one of the reasons machine for 
tests need some local modification - we may provide some Ansible-like testing 
script eventually.

But anyway - the easiest is to give us access to your test results so we could 
see whether there is something wrong with our test environment,  lvm2 bug, or 
system setup - it's not always trivial to guess...


Regards

Zdenek