Velocity testing or, as it is more commonly termed, testing velocity, is a metric for the speed with which all necessary activities can be set up and executed for a project – a software development project, in this case.
In this article, we will discuss about velocity testing, what it is in the context of agile, its importance and some strategies that can help improve the testing velocity the right way. So, let’s begin.
Table Of Contents
- 1 What is Velocity Testing?
- 2 What is Agile Velocity?
- 3 Why is testing velocity important?
- 4 What Goes Into High Velocity Testing?
- 5 Monitoring testing velocity
- 5.1 Need to Know: Speed vs. Velocity
- 6 Where Does High Velocity Testing Go Wrong?
- 6.1 When experimentation takes a backseat
- 6.2 Too much “Sample Pollution”
- 6.3 Focus shifted from Quality to Quantity
- 7 6 Strategies to Increase your Testing Velocity
- 7.1 Organize, gather, and report data
- 7.2 Run Parallel Tests
- 7.3 Keep the experiments small
- 7.4 Cut down time spent on regression testing
- 7.6 Embrace Automation
- 7.7 Velocity Testing Example
- 8 Conclusion
- 9 Frequently Asked Questions
- 9.1 What is high velocity testing?
What is Velocity Testing?
Basically, velocity testing measures how fast you can prepare and run development & testing processes in an SDLC. Velocity takes into account every aspect of the SDLC, right from ideation, research, requirements gathering, planning, and so on.
Along with volume (the number of activities in a project) and value (the rate of success for all such activities and experiments, velocity is one of the primary determinants of project success.
Needless to say, higher volume, velocity, and value – all contribute to boosting ROI.
Note: Though we’re not really discussing volume and value in this article, it’s important to remember that all three metrics are closely related. If you can improve project velocity, it will usually have a very visible effect on volume and value.
What is Agile Velocity?
Obviously, “velocity” is relevant for most industries that manufacture anything. However, in the context of modern software development, we’re looking at “Agile velocity”.
Agile velocity refers to the average measure of work a software development team does within an iteration or sprint, whatever their decided duration might be. It is a direct measure of how much gets done within each sprint. Deriving from this number, team leaders and management can estimate how fast a certain project or team will take to get to a desired level of productivity.
Agile velocity doesn’t just take into account how fast things are done. These things must also have a direct positive effect. For example, is more velocity helping to actually create more features and functionalities? Since customers don’t care about your process, that is the end result that really matters.
So, when talking about “velocity” within an Agile software development landscape, add “speed” to “result”. Doing random things fast does not translate to velocity. Doing the right things faster does.
Why is testing velocity important?
Testing velocity is essential because it reveals:
- How efficient the team is in setting up and executing development or QA activities within a project.
- Gaps in efficacy – could your team run more experiments than it currently is?
- Limitations in resource availability.
- Limitations in the bandwidth of team members to perform all necessary coding and QA actions
- The level of ROI being generated from the current velocity.
- The unlocked ROI that can be unlocked by optimizing velocity.
What Goes Into High Velocity Testing?
High velocity testing or high tempo testing is the idea that testing a lot of things very quickly (and breaking a lot of them too) is what leads to skyrocketing growth. This philosophy has been proved correct by major companies, especially tech-based companies like Facebook, Airbnb, Twitter (now X), Uber and more. Rapid experimentation and controlled risk taking directly contributed to them becoming the market behemoths they are today.
The logic is simple: the more you code, build, test and break things, the more you will be able to uncover your product’s inherent potential. For example, Facebook didn’t start out with a “Like” button, but when they added one, user engagement soared beyond even the creators’ expectations. Today, the “Like” or “Love” button on every popular social platform is essential. Not only is a higher number of “likes” valued because it shows the content’s popularity, but it can also translate to actual income – as it often does for influencers.
In many cases, companies that have used high velocity testing are said to be “growth hacking”. Turns now, doing more, building more, and breaking more was, in fact, their growth hack.
Monitoring testing velocity
To monitor velocity, you’ll have to look at a few major KPIs, and they aim to:
- Increase the number of tests run
- Increase the number of tests “won” – tests that get the job done
All the KPIs are meant to monitor velocity for purposes of growth, experimentation, and optimizing conversion rates.
To measure testing velocity, you look at the number of activities/experiments run within a particular duration of time. For example, let’s say you have a team of 10 developers, and it takes them each about one month to complete a feature. That translates to 10 features a month.
So, your current velocity is about 10 features a month.
Now, if you could find ways to optimize the process so that it takes a dev 2 weeks instead of a month to finish a feature, you can end up with 20 features a month. But, the process has to be optimized without placing undue stress and pressure on the developer. Tired, unhappy people cannot create great products or have great ideas.
Need to Know: Speed vs. Velocity
Generally, we use these terms interchangeably but they are not synonyms. Speed just means how fast something is going. Velocity, on the other hand, measures how fast an object or person is traveling in a specific direction.
Velocity requires direction, while speed does not.
So, when testing velocity, be mindful of getting things done in relation to the value they provide to your team, product, brand or customer. Just accomplishing a number of tasks at great speed doesn’t mean anything if it doesn’t improve the software or the development process.
When defining “velocity”, you have to measure the number of activities as well as their quality and contribution to the larger SDLC and business interests – traffic, MQLs, etc.
Where Does High Velocity Testing Go Wrong?
When experimentation takes a backseat
If you don’t fundamentally prioritize experimentation and data-driven growth, high velocity testing cannot exist. Thankfully, incorporating experimentation requires a few steps, as long as they are implemented consistently and supported by the entire team or company.
- Ensuring that all updates around optimization and insights are always shared with all the right people.
- Taking the effort to implement velocity-based optimization at all levels
- Accepting failure, especially in the early stages.
- Prioritizing learnings over success in the early stages.
To drive high velocity testing, you need a company culture that values experimentation. Folks should have the liberty to build fast, break fast, and fix fast. However, they must have the support of higher management to be able to do this adequately.
Too much “Sample Pollution”
If you’re running too many tests in too short a time (without the right preparation), you might end up with sample pollution.
This refers to any external factors, outside your control, that can meddle with test results and produce invalid data. This is normal for a lot of tests, but if you’re running too many of them too fast, your team does not have the time to slow down, find the issues muddying the test waters, and readjust your parameters.
So, in this case, the “velocity” is what destroys itself. By prioritizing speed over substance, the “high velocity testing” yields no results and stops being “high velocity”.
Focus shifted from Quality to Quantity
If you shift focus from quality to quantity – from the effectiveness of tests to the number of tests and how fast they are run – high velocity testing cannot exist. For things to stay “high velocity”, both quality and quantity must be at optimal levels simultaneously.
Here’s a quote that we cannot remember the source of, yet it is one that accurately encapsulates this point, “To be frank, it doesn’t matter how quickly you test bu****** ideas”.
The metrics that matter here are:
- Are the tests effective? Are you running tests that actually provide value to the SDLC?
- What is your average “win” rate? As in, how many effective tests are being run to completion, and providing appropriate tests?
- Are you ascribing sufficient resources to the right tests?
- Are the changes and optimizations you’re implementing actually leading to improvements over time?
A few other circ*mstances where high velocity testing does not work:
- Budgets for optimization and experimentation are limited.
- Team size is not large enough to run enough tests.
- If your team is not skilled enough (yet) with regard to optimization and experimentation targeted at raising test velocity.
6 Strategies to Increase your Testing Velocity
Organize, gather, and report data
Start by gathering data on your existing processes. The first question you need to answer is “What is our current testing velocity?”
Establish the “starting point” of your experiments by looking at current activities in the SDLC. How else would you know if your efforts are actually leading to higher velocity?
Next, identify the parts that need optimization, and start reporting on the results of such optimization. Report and document everything. This keeps your goals highlighted in your mind, monitors progress consistently, and lets you maintain focus on making changes that drive value.
Run Parallel Tests
Use parallel tests to increase testing velocity. It is the simplest way to do so.
In parallel testing, you run multiple tests simultaneously. This is especially effective when running cross-browser and cross-device tests. If the software has to be tested on hundreds of browsers, devices, browser versions, OSes, and different versions of each OS, then you need parallel testing (and automated testing in general) to meet your deadlines.
Naturally, the number of valuable tests you can run in a restricted period will increase in number, because you are running multiple tests at the same time.
Keep the experiments small
Big experiments don’t always result in big improvements. Sure, certain experiments need to be designed with complex design and parameters, but that is not the rule.
In the real world, larger experiments, on average, have the same impact as smaller ones. Therefore, whenever possible, build smaller experiments/tasks for optimization.
Not only do they require less research, planning, design, and deployment, but they also run faster, which means you can run more of these experiments. You save time and get more results. In the long run, this will actually lead to wider change in your systems – the kind of changes that actually translate to increased conversions.
Cut down time spent on regression testing
Yes, regression tests are essential to developing and releasing software. This point doesn’t ask you to compromise them or get rid of them. It asks you to design smarter regression tests that can be finished faster.
This is because regression tests take up a significant amount of time, and must be run after every code change. So, if they do not run in the shortest possible time, the entire SDLC is paused multiple times during a single project. Remember, you can’t make code changes while a regression test is running.
Naturally, shortening the time used by regression tests by writing more targeted scripts (based on clear scenarios, provisioning high coverage and low maintenance) will shorten the entire pipeline, and increase testing velocity.
Test on Real Browsers & Devices
There is no way you can get 100% accurate software testing results without testing on real browsers and devices. And, if you do not get 100% accurate test results, you cannot trust the software to be released to public users. If you do release it, expect user backlash due to the bugs that will inevitably show up.
So, if you do not run software tests on real browsers and devices, you will either have to test them again (on real browsers and devices) before release…or you deal with buggy software that no one likes.
Taking the former option will obviously slow down velocity because you’re basically running the same tests twice. Instead, why not just start off by testing on real browsers and devices, so you know exactly how the software will behave in the hands of real-world users?
Note: Emulators, simulators, and virtual machines are fine for testing in the early stages of the SDLC. They, however, are not suitable to test the software for prod release.
Embrace Automation
With 1 or 2-week sprints becoming the norm, no SDLC can survive without automation. Simply put, manual testing is not fast enough to push velocity anywhere close to the levels it must achieve to succeed in modern markets.
It does not matter what your experiments are; a large part of them needs to be automated. Not only are automated tests faster, but they are also ideal for running repetitive tests. It can also take care of administrative tasks the team must complete.
By automating as much of the SDLC as possible, testers will find themselves with more time to develop innovative experiments that drive better quality and profits. You can start with small-scale automation, and keep focusing on incremental changes. When you can shave precious minutes or hours off each test through automation, the time and effort saved will add up significantly in the end.
Velocity Testing Example
- Set the benchmark
When you start a sprint, assign a certain number of user stories to the team. Let’s call it 5. The team gets 5 points for each story they complete. Completion of each story also includes testing and automating it.
In this example, we assume that the team has completed 4 out of 5 stories. So, they get a total of 20 points.
- Answer “How many sprints do we need?”
Now that you know your team’s velocity, you can determine how many sprints will be required to complete the project. Basically, this is when you revise predictions about timelines.
In this example, each sprint allows for 4 user stories. Let’s say the project needs 40 stories to be accomplished for public release.
Therefore, the team will need 10 sprints to complete the project.
Conclusion
If you don’t know where you stand, you cannot improve. If you don’t know your team’s current velocity, you cannot optimize it for better results.
Velocity Testing is basically “Know Thyself” (thanks, Immanuel Kant) for software testing. It is absolutely mandatory, and instrumental to refining and streamlining the processes with the modern SDLC. Since Agile’s success is defined by a pipeline speed (without compromising quality), Velocity Testing must be a steady part of any conversations around planning, strategizing, and brainstorming software development, design, and testing.
Frequently Asked Questions
What is high velocity testing?
High velocity testing refers to a scenario in which a team/company can maximize its development/testing activities. This is done by running a large number of meaningful experiments within a specific period of time (usually a sprint).
The more activities you execute, the more ideas you test out and the more you learn about improving software quality and audience preferences.
