Automated tests are used to validate features in development environments but also in production. Whereas the classic approach of keeping all tests in the same code project is the most popular, it is not the best idea (and by code project i mean for example a Maven project).
In my opinion neither is keeping the tests in the same code project but in different modules or packages. Here are a few reasons why you should keep your production and development environment tests in separate code projects:
- No more environment configuration hell: By having your test code separated into two repositories, you eliminate checks and configurations that look like: “if this is the production environment behave in this way, else behave in another way”. In general tests should not be “aware” of which environment they are running on. They should not really contain any environment configuration. Switching environments should be independent from the test code. This way tests will also be shorter and cleaner.
- Dependency management: It might be that in development environments you will test against some versions of certain libraries/dependencies that you do not have access to in production, at least for a while, until the use of these dependencies is feasible for the production code. Therefore, since you cannot have two versions of the same library in your project, you cannot run the same test on both dev and production environments. Or at least not easily.
- No more “oops did i just run tests in production” moments: Constantly switching between environments while running tests might, at times, become confusing. You might, by mistake, forget that you had setup the tests to run in production, and you might run them even if you did not want to.
- No accidental validation in development environment: Even worse, if you meant to test a feature change in a development environment to confirm it works properly, but forgot to change the environment from production to the correct development one, you might falsely think the feature under test works properly.
- Different classes with same name in same code project (but in different modules or packages) might lead to accidentally updating the tests you didn’t mean to update.
- Light setup for production tests: the project that will contain only production tests will not need plenty of the dependencies that tests for development environments do. Basically, a good portion of mocking related dependencies and data will not be needed.
- Tests for production should only be a subset of all tests required to validate a feature in the development environment. There you want to check for all kinds of scenarios, but when it comes to production – your features should already be validated through plenty of scenarios that were run in previous environments.
Bottom line is: if you keep your production tests separate from those dedicated to your development environments, you avoid accidentally altering or running them, and you have a clear delimitation of what tests are needed where.