[Avocado-devel] calling self.fail() in setUp() == ERROR result?

[Lukáš Doktor]

Dne 12. 09. 19 v 13:50 Brian J. Murrell napsal(a):
> In our case, we are testing a client-server protocol.
> 
> All of our tests need the server to be started up, which is ripe for
> putting into setUp().
> 
> But a failure to start the server could as likely (more, probably) be a
> problem in the server start-up code, which is a test FAIL because it
> should be sent to developers, not QA/CI guys.
> 
> Sure, we could move server start-up back into to every test, but that
> means repeating the same code over and over again across every test.
> Such a thing really invalidates the point of setUp(), IMHO.
> 
> Cheers,
> b.
> 

Well purely academically speaking, there should be one test that starts the server, therefor start of the server should be part of the "test" there and in case of failure, it should FAIL. Then there should be bunch of tests that start the server in "setUp" and CANCEL the execution in case it fails to start the test as environment is not ready.

Now in real world (again, coming from virt background) I'd run the "start server" test prior the actual testing and based on avocado return I'd simply not execute the following set of tests (in case the same/similar server is started for each test). If each test uses different options and it's likely that some variants work, some fail, I'd simply use setUp to start the server as you mentioned. Anyway the setUp failure would still be environment issue and not really a test failure.

Anyway I don't have strong opinion about this, but IIRC @Cleber had. So let's hear his arguments...
Regards,
Lukáš

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20190912/aaee1851/attachment.sig>