[sos-devel] Proposition to submit Ubuntu's autopkgtest integration script for upstream inclusion

[Louis Bouchard]

Hello,

In order to have the latest sosreport version included in all releases
of Ubuntu, I have developed two scripts that are automatically executed
when a new sosreport package is uploaded to the Ubuntu archive. They
have been quite useful so far in detecting bugs that went unnoticed
before they hit the ubuntu archive. Both script use the unittest
infrastructure and are written in python3.

The scripts can be found here :

https://anonscm.debian.org/cgit/collab-maint/sosreport.git/tree/debian/tests

assert.sh is unused and will be removed. control has autopkgtest's
specific instructions and is outside of the scope of this discussion.
check-mandatory-files and mandatory-files is the start of another test
but is unused at this time.

sos-run-nonroot.py and sos-run.py are the two scripts that I am talking
about.

I have no clear idea on where they would fit, but here is what they do.

sos-run-nonroot.py
==================
The script will run 'sosreport --batch --tmp-dir {some work dir}' and
verify that no plugin is actually enabled. It will also pick up the sos
version advertized and verify it against the version in sos.spec. This
was useful to catch a few glitches in the debian pkg build process.

Any kind of failure in this script will cause the execution to fail.
autopkgtest will see that as a show-stopper and block the package from
the archive.

sos-run.py
==========
The class setUpClass will run 'sosreport --batch --tmp-dir {some work
dir}'. It will then try to find the tarball in the working directory.

test_extract_tarball() will extract the content of the tarball
previously created if it exists. Failure will occur if no tarball is
found or if problems have been encountered while extracting the tarball.

test_check_run_errors() will verify in the extracted tarball if there is
*errors.txt files in the ./sos_logs subdirectory. If so, it will fail
the test and indicate where to look for errors. Error files will also be
added as autopkgtest artifacts.

test_md5_checksum() does a verification on the validity of the md5
checksum calculated when sosreport was executed against the real tarball
created.

It should not be overly complex to dissociate the autopkgtest specifics
and make those tests more generic. This is what would be included upstream.

Please let me know what you think of all this.

Kind regards,

...Louis