I develop my current project with a very strict approach to TDD and try to make it as close to Kent Beck’s TDD By Example book. So right now I’ve got code coverage of 93%, with 1290 unit test cases, 58 integration tests, and 25 UI tests. However, the tests durations on the CI are too long IMHO:
- Unit tests: 15 minutes
- Integration tests: 10 minutes
- UI tests: 30 minutes
And this will add up when I need to checkout and compile the dependencies from the dependency managers (Cocoapod, Carthage and SPM). So much so that it was canceled by the CI because it took too long. Can you suggest me a good way to approach the testing?
A few facts and questions about the project:
I separate the tests by the effects of the method. For example, consider a method called
loadData(), this will have these test cases:
Is this the right approach? Or can it be simplified more?
I’ve set the timeout for integration and UI tests to 5 seconds. At first, I use 2 seconds but it returned lots of false negatives because of timeouts on the CI. These tests got red almost 20-30% of the times with 2 seconds timeout. From your experience, what is the proper timeout that you usually use?
I’ve got 4 main flows in the CI:
- Development: triggered when there’s a push to any branch besides master
- Pull Req: triggered when there’s a pull req update
- Master: triggered when there’s a merge to master (ie. when a PR is merged)
- Store: triggered manually when we want to build for the store
All of these flows always run all the tests (ie. unit, integration, UI). Should I disable some of the tests on a few of the flows? I was thinking that the Development flow only need to test for unit tests, and the rest can be run in Pull Req flow, while Store will only need to test for UI tests? Or something else.