In our last article, we talked about the three main impacts to high velocity: methodology, environment and application stability. Now that we understand what impacts high velocity, we can dive into the ways automated testing is important to achieving and maintaining it.
One of the cornerstones of high velocity is keeping the developers focused on the feature at hand, versus fixing defects uncovered in previously developed features. The best way to achieve this is by regression testing, ideally at each build, but nightly at a minimum. When the first set of features in an application is developed, all of the features are manually tested. No regression testing is required because nothing has been integrated. However, once the first few features are integrated, you have a baseline working application that must be kept in good working condition. The only way to maintain this is to perform a regression test of the new features with the remaining application.
When Should Regression Testing Occur?
Regression tests are relatively easy to perform early in the application’s life. Just manually execute the previous features’ functional tests before the end of the sprint. As the number of features grows, though, it becomes impossible to manually execute all of the previous functional tests in the time allowed. At this point, you need to consider automating your regression tests. Automating regression tests allows you to reduce hours or days of manual testing to just minutes or a few hours, allowing regression testing at build time or nightly.
Initially, automated regression tests are designed to run serially. That’s because it is relatively easy to design serial automated tests. However, as your application grows, you will eventually exceed your execution time window for post-build or nightly execution. At that point, you need to redesign the tests so that independent tests are executed in parallel. This gets the regression test time back to an acceptable duration to allow regression testing either at build time or nightly, giving developers immediate feedback on regression test defects.
3 Keys to Designing a Regression Test Suite
When designing a regression test suite, it is important to focus on three things: criticality, stability, and length. First, choose those regression tests that exercise the most critical application features, or the business transactions that give the greatest coverage of a typical user’s work. Second, choose regression tests that test relatively stable parts of the application. Features that are in a constant state of flux are poor candidates for regression testing, as the automated regression test would rarely be in sync with the feature.
Third, choose regression tests that are fairly lengthy and take a long time to execute manually. These are prime candidates for automated testing. It doesn’t make sense to automate a test that takes a couple of minutes, so look for tests that take hours to execute manually or require a variety of different data combinations to fully test the feature.
Now that you’re familiar with the valuable relationship between regression testing and high velocity, we’ll focus next on shifting your defect detection upstream in your development process.
