Once you have a regression suite set up, you will need to run it. When you have a smaller number of tests that need to be run on a specified day, that won’t be a problem, and the tests will successfully finish running within the allocated time period. However, as the suite becomes larger and larger, so does the amount of time required to run all the tests. In some cases, you can even get beyond a 24-hour test run. So, what can you do to optimize the test run time? You can use two strategies: parallelize and split. Continue reading The Automated Regression Suite. Part 3 of 3. How to run the suite
Once you have your automated regression suite in place, you can create a scheduler to run them periodically, without any manual intervention. Mostly you will use Jenkins jobs (or some similar CI tool) to trigger them and have them running on an environment of your choice. Just because they are called “regression tests” it does not mean they are only meant to be run once before a release. They are in place to help validate your system, so you can run them as often as you want. Continue reading The Automated Regression Suite. Part 2 of 3. When to run the tests.
What do I mean by “automated regression testing”? I am not one for debating for hours what this means, so let me give you my interpretation (not definition), so that we are on the same page: whenever you are performing a new release, you need to make sure the features you released some time ago still work properly. For that, you will need to run some kind of tests, to ensure the features are still working as expected. You could do that manually, but running the same manual test cases repeatedly, for each release, takes a lot of time and quite frankly, becomes boring or even frustrating at one point. Hence, the suite of automated tests comes in handy. Having these in place will allow to verify plenty of scenarios while you can do something more enjoyable during the test run. Continue reading The Automated Regression Suite. Part 1 of 3. When to create the tests for regression
Writing automated tests is no longer the biggest challenge in the testing community. Writing reliable automated tests is. So many times tests that were once written are sent to the garbage bin or thrown into oblivion. They are unreliable and people will just ignore them when they are running, simply because they have a history of failing for various random invalid reasons. Continue reading Write automated tests with repeatable results
This is something i heard quite a few times: why bother writing automated tests, as they will not find any bugs anyway? Continue reading A false myth: automated tests don’t uncover bugs