[libvirt] libvirt TCK wrapper for autotest review

Lucas Meneghel Rodrigues lmr at redhat.com
Wed Feb 22 20:45:58 UTC 2012


Hi guys,

I was here looking at the autotest wrapper for libvirt TCK and then 
decided to work on it, as I had the review fresh on my mind. Things that 
I've worked on:

* Fixed some download links, that were already sent to upstream tck and 
applied (thanks Dan Berrange)
* Instead of making all tests output to the same DEBUG log, make them 
output to separate .tap files on the results directory
* Run all tests available for a given item, rather than stopping the 
test on the first failure
* removed capitalization on the wrapper name, since it's project policy
* Use os.environ, and some features of the subcommand execution API to 
execute the tests
* Remove usages of error.JobError, as the problems there are more 
error.TestError, since they are restricted to the libvirt_tck test, not 
the entire job (in autotest, a job can do more stuff than just a 
sequence of job.runtest() calls).
* Made the error messages more descriptive, with info of all failed tests

So, the current output of the tests is like this:

$ sudo client/bin/autotest run libvirt_tck
18:29:27 INFO | Writing results to 
/home/lmr/Code/autotest.lmr/client/results/default
18:29:27 INFO | START	----	----	timestamp=1329942567	localtime=Feb 22 
18:29:27	
18:29:27 INFO | 	START	libvirt_tck.domain	libvirt_tck.domain 
timestamp=1329942567	localtime=Feb 22 18:29:27	
18:30:19 ERROR| child process failed
18:30:19 INFO | 		FAIL	libvirt_tck.domain	libvirt_tck.domain 
timestamp=1329942619	localtime=Feb 22 18:30:19	FAIL: 
['120-disks-stats.t', '205-disk-hotplug-ordering.t']
18:30:19 INFO | 	END FAIL	libvirt_tck.domain	libvirt_tck.domain 
timestamp=1329942619	localtime=Feb 22 18:30:19	
18:30:19 INFO | 	START	libvirt_tck.hooks	libvirt_tck.hooks 
timestamp=1329942619	localtime=Feb 22 18:30:19	
18:30:19 ERROR| child process failed
18:30:19 INFO | 		FAIL	libvirt_tck.hooks	libvirt_tck.hooks 
timestamp=1329942619	localtime=Feb 22 18:30:19	FAIL: 
['051-daemon-hook.t', '052-domain-hook.t']
18:30:19 INFO | 	END FAIL	libvirt_tck.hooks	libvirt_tck.hooks 
timestamp=1329942619	localtime=Feb 22 18:30:19	
18:30:19 INFO | 	START	libvirt_tck.networks	libvirt_tck.networks 
timestamp=1329942619	localtime=Feb 22 18:30:19	
18:30:28 INFO | 		GOOD	libvirt_tck.networks	libvirt_tck.networks 
timestamp=1329942628	localtime=Feb 22 18:30:28	completed successfully
18:30:28 INFO | 	END GOOD	libvirt_tck.networks	libvirt_tck.networks 
timestamp=1329942628	localtime=Feb 22 18:30:28	
18:30:28 INFO | 	START	libvirt_tck.nwfilter	libvirt_tck.nwfilter 
timestamp=1329942628	localtime=Feb 22 18:30:28	
18:30:32 ERROR| child process failed
18:30:32 INFO | 		FAIL	libvirt_tck.nwfilter	libvirt_tck.nwfilter 
timestamp=1329942632	localtime=Feb 22 18:30:32	FAIL: 
['090-install-image.t', '100-ping-still-working.t', 
'210-no-mac-spoofing.t', '220-no-ip-spoofing.t', 
'230-no-mac-broadcast.t', '240-no-arp-spoofing.t', '300-vsitype.t']
18:30:32 INFO | 	END FAIL	libvirt_tck.nwfilter	libvirt_tck.nwfilter 
timestamp=1329942632	localtime=Feb 22 18:30:32	
18:30:32 INFO | 	START	libvirt_tck.qemu	libvirt_tck.qemu 
timestamp=1329942632	localtime=Feb 22 18:30:32	
18:30:40 ERROR| child process failed
18:30:40 INFO | 		FAIL	libvirt_tck.qemu	libvirt_tck.qemu 
timestamp=1329942640	localtime=Feb 22 18:30:40	FAIL: 
['205-qcow2-double-backing-file.t']
18:30:40 INFO | 	END FAIL	libvirt_tck.qemu	libvirt_tck.qemu 
timestamp=1329942640	localtime=Feb 22 18:30:40	
18:30:40 INFO | 	START	libvirt_tck.selinux	libvirt_tck.selinux 
timestamp=1329942640	localtime=Feb 22 18:30:40	
18:30:49 ERROR| child process failed
18:30:49 INFO | 		FAIL	libvirt_tck.selinux	libvirt_tck.selinux 
timestamp=1329942649	localtime=Feb 22 18:30:49	FAIL: 
['055-dynamic-base-label.t', '100-static-relabel-no.t']
18:30:49 INFO | 	END FAIL	libvirt_tck.selinux	libvirt_tck.selinux 
timestamp=1329942649	localtime=Feb 22 18:30:49	
18:30:49 INFO | 	START	libvirt_tck.storage	libvirt_tck.storage 
timestamp=1329942649	localtime=Feb 22 18:30:49	
18:31:24 INFO | 		GOOD	libvirt_tck.storage	libvirt_tck.storage 
timestamp=1329942684	localtime=Feb 22 18:31:24	completed successfully
18:31:24 INFO | 	END GOOD	libvirt_tck.storage	libvirt_tck.storage 
timestamp=1329942684	localtime=Feb 22 18:31:24	
18:31:24 INFO | END GOOD	----	----	timestamp=1329942684	localtime=Feb 
22 18:31:24	

As time allows, I might take a look at the failures and help with 
libvirt_tck.

I've combined all modifications to a single, self contained commit. 
Also, as the work is self contained, it could be very very easily 
rebased to the latest upstream tree. I've updated my personal repo and 
sent a github pull request that you guys can see and review:

https://github.com/autotest/autotest/pull/192

I'm still not going to merge this to the upstream tree just yet, since 
I'd like to hear some feedback from you guys. As for developing together 
with autotest, I guess you can easily use the clone you have on libvirt 
now, and on your working directory you can add the following remote:

[remote "upstream"]
         fetch = +refs/heads/*:refs/remotes/origin/*
         url = git://github.com/autotest/autotest.git

Then from time to time, you can just pull from upstream to your master 
branch:

git pull upstream master

And then rebase your development trees:

git checkout tck
git rebase master

It should be painless, given the fact that the wrapper is pretty much 
self contained, and isolated from the rest of the code.

Please let me know what you think,

Lucas




More information about the libvir-list mailing list