Once you have a regression suite set up, you will need to run it. When you have a smaller number of tests that need to be run on a specified day, that won’t be a problem, and the tests will successfully finish running within the allocated time period. However, as the suite becomes larger and larger, so does the amount of time required to run all the tests. In some cases, you can even get beyond a 24-hour test run. So, what can you do to optimize the test run time? You can use two strategies: parallelize and split. Continue reading The Automated Regression Suite. Part 3 of 3. How to run the suite
Tag Archives: testng
Better Test Code Principles: #2 Don’t generate ALL your test data in @BeforeClass
A considerable amount of tests will need some test data to be generated previous to them running. Some people prefer to put all the data creation for all the tests in a class into the @BeforeClass method, some others prefer to keep the prerequisite data creation inside the tests themselves. Continue reading Better Test Code Principles: #2 Don’t generate ALL your test data in @BeforeClass
Share this:
SoftAssert – don’t make your test fail on the first assertion failure
When you have more than one assertion in your test, you might want one of two things:
- Have your tests fail once the first assertion failure is encountered.
- Have all your assertions run, no matter if they have passed or failed. Of course, after they are run, if there are failures, you want the test to fail, and also show you where the issues were.
Continue reading SoftAssert – don’t make your test fail on the first assertion failure
Share this:
Quick Tip: Running automated tests in parallel
The What
I have a bunch of tests that i would like to run faster, by making them execute in parallel. In my tests:
- I am not using a DataProvider and only want to make the same test run several times.
- I am using a DataProvider and want my test to run with the provided values from the provider, but in parallel.
- I am not using DataProviders, but my tests are ran by using the textng.xml file that specifies which tests to run (as per this article).
Continue reading Quick Tip: Running automated tests in parallel
Share this:
DataProviders for TestNG tests
When you need to run the same test a number of times, by changing only a few variable values, instead of writing several identical tests, you can use the dataProvider functionality offered by TestNG. What you have to do is declare a dataProvider method that returns a list of lists of objects, pass it to the test method, change the test method’s signature to correspond to the types of objects found in each inner list and run the test. Continue reading DataProviders for TestNG tests
Share this:
TestNG custom listeners: ITestListener
When running TestNG tests, one could want to perform some common actions – after each test has finished successfully, after each failed test, after each skipped test, or after all the tests have finished running, no matter their result. To apply such a common behavior to a group of tests, a custom listener can be created, that implements TestNG’s ITestListener interface. There are two types of methods to implement: a set of them are related to a single test’s run (onTestStart, onTestSuccess, onTestFailure, onTestSkipped, onTestFailedButWithinSuccessPercentage), the other to the whole suite’s run (onStart, onFinish). These methods, depending on their type, have a parameter passed to them: result, which is of type ITestResult – for the test run, and context, which is of type ITestContext, for the suite run. These types offer different information, for example: ITestResult can tell you when a test began to run, when it finished running, what status the test had (whether failed, passed), whereas ITestContext can tell you the list of all the passed tests, all the failed ones, and so on.
Continue reading TestNG custom listeners: ITestListener
Share this:
Running TestNG tests
Running TestNG tests can be done in two ways: either directly from the IDE (by selecting the desired tests and choosing to ‘Run TestNG tests’) or from the command line. The latter option is very useful when trying to run only a selection of all the tests, that might spread across different classes or packages, or to run tests that belong to certain groups. To do this, you need to create some .xml files that will select the tests that need to run and/or exclude tests not to run, add a configuration in the Maven profile created for running the tests, and run the proper Maven command. Continue reading Running TestNG tests
