Suppose we need to ensure that a sequence of database operations – currently implemented as A followed by B followed by C – executes either all together, or not at all. We can achieve this using a database transaction – but how can we write an automated test to verify that this atomicity property is satisfied, which is not brittle?
We can fairly easily write a white-box test for this: given that we know that A is executed first, then B, then C, we can write a test which injects a fake C and makes the fake C throw a runtime exception instead of issuing a database update query, to simulate a system crash. Then we catch that exception in the test, and assert that nothing in the database has changed since the test began running.
However, if a developer later comes along and rewrites the implementation to call C first, our test will become invalid – it will only test that “if nothing happens, then nothing happens” or in other words “if the system crashes immediately, then nothing will be written to the database”. Not a very useful test, and not testing what we want to test! This is because it’s a white-box test which is predicated on an assumption about the order of operations inside the implementation – namely, that C is executed last (or at least, not first).
Is it possible to avoid the test turning into a bogus test in this way if the order of database operations in the implementation is changed?