imalittletester

After the project has been created, you will need to decide how you want your automated tests to run. Keeping in mind that developers write unit tests, which by definition will validate of the code by itself, without interaction with other components, they are suitable to validate that the code commited satisfies the requirements in isolation. They should run fast, and not need interaction with browsers for example.

On the other hand, acceptance tests, which are the tests written by QAs should validate the code in the actual environment where it will reside, having contact with all the components around it. These tests validate that the code still acts properly when it runs in the system that it is built in. For that reason, these tests might take a long time to run, use browser instances, and might be rather fragile when it comes to succeeding. For example, Selenium tests are some of the most fragile, since, if you run tests in some sluggish environment, they might fail because of the slow responsiveness of the environment, pages not loading on time, and so on. Hence, running these tests for every project compilation phase is not feasible, as the build might never compile successfully.

To sum it up: you will want some tests to run when the build is compiling, to get an actual build out on the market; you will want these tests to run, to fail if they find some issues, but not to stop a build from being made, if the build is not broken. First of all, the situation described here applies to building the project with a Maven command (‘mvn clean install’ – which by itself will or will not run automated tests, as described below). To run tests but not within the building phase of the project, you will use another Maven command, with the proper configurations (which is ‘mvn test’). Running TestNG tests from within Maven is not possible without setting up a plugin in the pom.xml file, called maven-surefire-plugin. Just like other dependencies that you will place in the pom file, this plugin is to be specified by entering the groupId, artifactId, version, which can be found, as described in an earlier post, in the Maven repository. To be able to distinguish between unit tests (developer tests) and QA tests, it is recommended for each of these tests to have a @TestNG ‘group’ associated with them (@TestNG annotation attributes are described in this post: https://imalittletester.com/2014/02/27/testng-test-attributes/). You would have a group for your unit tests (suggestion : “unit_test”), and a different one for the acceptance test (suggestion: “acceptance_test”). You will therefore configure the surefire plugin to run all the test that have the “unit_test” group associated at project build, and run all the “acceptance_test” tests at a different phase, namely the Maven test one.

Step one: compiling the project. At this phase, the ‘mvn clean install’ command will be used and the unit tests will be run.  This command not only compiles the code and runs the specified tests against the compiled code, but it also installs the package that results from compilation into the local repository. The following entry needs to be inserted into the pom.xml file of the project:

 <build>
  <finalName>theProjectName</finalName>
  <plugins>
   <plugin>
   <groupId>org.apache.maven.plugins</groupId>
   <artifactId>maven-compiler-plugin</artifactId>
   <configuration>
      <source>1.8</source>
      <target>1.8</target>
   </configuration>
   </plugin>
   <plugin>
     <groupId>org.apache.maven.plugins</groupId>
     <artifactId>maven-surefire-plugin</artifactId>
     <version>${maven.surefire.version}</version>
     <configuration>
        <excludedGroups>acceptance</excludedGroups>
     </configuration>
   </plugin>
   </plugins>
 </build>

Looking at the configuration, the following are to be specified:

• source and target: the Java version on which the project will run
• excludedGroups: the groups associated with tests that are not to be run at compile time. In this case, all the tests that have the ‘acceptance’ group are ignored at this step.
• Notice that there are two plugins declared within the build section. The first one, compiler, is used to compile the project, while surefire is the one that will run the tests.
• It is worth mentioning that, when using the ‘excludedGroups’ attribute, all classes that are @Test but have no group associated will be run at this phase. To avoid this, add the proper group to these tests.

Step two: run the acceptance tests. You can run the tests from the command line. You will create what is called a ‘Maven profile’ (which has a few settings which i will describe in a second), and you will run the following command from the command line: ‘mvn test -Pacceptance ‘.

 <profiles>
  <profile>
  <id>acceptance</id>
  <build>
   <plugins>
    <plugin>
     <groupId>org.apache.maven.plugins</groupId>
     <artifactId>maven-surefire-plugin</artifactId>
     <version>${maven.surefire.version}</version>
     <configuration>
       <excludedGroups>unit</excludedGroups>
     </configuration>
    </plugin>
  </plugins>
 </build>
 </profile>
 </profiles>

Within this code section, you will define a profile, named ‘acceptance’, that will be used for running all the tests that do not have the ‘unit’ group associated with them. The part of the command ‘-Pacceptance’ is the one that specifies that a profile will be used for running the tests against the profile’s settings.

When would you use these commands? Well, for the first step, whenever new code is committed into the version control system. The second one, as soon as a build is deployed into a test environment.

Regarding the profile that was created here, there is another addition that can be put in place. If you want to run only a selection of the acceptance tests that exist within a project, you can create a set of xml files that will select only those tests, and you will provide the xml files as parameters to the ‘mvn test’ command. More on that here:  https://imalittletester.com/2014/03/01/running-testng-tests/