[Avocado-devel] calling self.fail() in setUp() == ERROR result?

[Lukáš Doktor]

Dne 11. 09. 19 v 15:38 Brian J. Murrell napsal(a):
> If I call self.fail() in a test_* method, the test reports FAIL as I
> would expect.
> 
> If I call self.fail() from setUp() (or presumably tearDown()) the test
> reports ERROR.
> 
> Is this intended behaviour?  Am I not able to have a test report FAIL
> from setUp() or tearDown()?
> 
> Cheers,
> b.
> 

Hello Brian,

this is expected behavior. It's because the test hasn't actually started, therefor it can not fail. The only possible interruption on failed setUp are:

1. Skip (no tearDown)
2. Cancel (tearDown)
3. TestSetupFail (reported as ERROR and tearDown is executed)

This is deliberate as setUp issues should not be related to the module under testing. It is related to the environment, therefor the test can be interrupted (Cancel) or pre-conditions are not met and it is not even set (Skip). In case of a failure, the module-under-testing was not exercised, which leaves us only with ERROR.

Anyway feel free to share more details if you want us to re-evaluate, but this helped us to distinguish between module-under-testing failures vs. setup issues. (and to actually get people to write better tests)

Regards,
Lukáš

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20190911/e6569f34/attachment.sig>