<div dir="ltr">Hi Thomas,<div><br></div><div>I ran into this problem a while back and though I though I had it fixed at least for my work item type black box tests [1].</div><div><br></div><div>I've been using the "suite" package [2] from the testify repository. You have to move your TestXXX functions to methods of a struct with additional special functions (your suite struct). These allow you to guarantee certain preconditions are met before and after any of the tests is run (SetupSuite, and TearDownSuite). If you need special preconditions to be fulfilled before or after an individual test is run, you can also ensure that with two methods (SetupTest, TearDownTest).</div><div><br></div><div>I'm pretty sure that those special methods will be executed even if a test fails.</div><div><br></div><div>All in all, this should elegantly solve the problem of having a "clean" database before and after a test is run, right? In my test [1] I blindly remove the DB entries, that I want to create in the test, before and after a test is run.</div><div><br></div><div>Regards,</div><div>Konrad</div><div><br></div><div>[1] Work item type black box tests: <a href="https://github.com/almighty/almighty-core/blob/fd204a0420e4a454832b6867f2b556d244f7f646/workitemtype_blackbox_test.go#L97">https://github.com/almighty/almighty-core/blob/fd204a0420e4a454832b6867f2b556d244f7f646/workitemtype_blackbox_test.go#L97</a><br></div><div>[2] Suite Package: <a href="https://github.com/stretchr/testify#suite-package">https://github.com/stretchr/testify#suite-package</a></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Oct 27, 2016 at 8:28 AM, Dhriti Shikhar <span dir="ltr"><<a href="mailto:dshikhar@redhat.com" target="_blank">dshikhar@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Tuesday 25 October 2016 09:09 PM, Thomas Mäder wrote:<br> <br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Hi folks,<br> <br> in PR <a href="https://github.com/almighty/almighty-core/pull/294" rel="noreferrer" target="_blank">https://github.com/almighty/al<wbr>mighty-core/pull/294</a> I have used the pattern to roll back to execute tests in a transaction which is rolled back at the end of the test. Thus, the test is guaranteed to not change the database. There were couple of objections to this patterns, which leads me to seek some convergence on the topic of db tests.<br> </blockquote></span> This approach works well and it is quite effective in isolating changes between tests. Except in the case where you need to test the transaction itself.<div class="HOEnZb"><div class="h5"><br> <br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <br> Every test has certain preconditions. For example, when I test creation of an object with a given ID, the precondition is that this ID does not yet exist in the DB. If the preconditions are not met, the test cannot properly run and thus does not yield any information about the code it is supposed to test.<br> <br> To solve this problem, there are two approaches. The first one is to program the test without having preconditions. For example, instead of creating an object with the ID "42" in the above case, the test could try to use a UUID as the ID for the object it creates. Since there will be no collision with previous runs, there is no precondition that can be violated.<br> <br> The second one is get the system into a well know state before the test runs. One of the patterns to do this is to always start with an empty database. Alternatively, the test setup could contain code to establish the precondition. In our example, the setup would delete the object with the ID "42" (if it exists) in order to make sure that the create call can be tested.<br> <br> One of the problems we have is that we run unit often against a local database while developing. Even if the tests are written to work repeatably, we don't want the test database to be trashed every time nor filled up with useless test data. So ideally, tests should clean up after themselves.<br> <br> The rollback pattern keeps the db from filling up, but can't prevent failed preconditions. Another pattern is to setup objects used in the test before the test and to remove them after the test. However, this doesn't really work either: tear down may fail and leave objects in the database. The next run of the test may then fail.<br> <br> I think the only reliable way to write db tests is this:<br> <br> setup<br> <br> delete all objects the test expects NOT to exist<br> <br> delete all objects the test expects to exist (because they may not be in the correct state)<br> <br> create all objects the test expects to exist.<br> <br> Test<br> <br> use the objects from above<br> <br> tearDown<br> <br> remove all objects created in setup or the test<br> <br> What are your proposals on how to structure db tests?<br> <br> /Thomas<br> <br> ______________________________<wbr>_________________<br> almighty-public mailing list<br> <a href="mailto:almighty-public@redhat.com" target="_blank">almighty-public@redhat.com</a><br> <a href="https://www.redhat.com/mailman/listinfo/almighty-public" rel="noreferrer" target="_blank">https://www.redhat.com/mailman<wbr>/listinfo/almighty-public</a><br> </blockquote> <br> ______________________________<wbr>_________________<br> almighty-public mailing list<br> <a href="mailto:almighty-public@redhat.com" target="_blank">almighty-public@redhat.com</a><br> <a href="https://www.redhat.com/mailman/listinfo/almighty-public" rel="noreferrer" target="_blank">https://www.redhat.com/mailman<wbr>/listinfo/almighty-public</a><br> </div></div></blockquote></div><br></div>