What are the common pitfalls and mistakes to avoid when estimating testing time and resources? (2024)

(edited)

I find that, often in these estimation techniques, human factor is rarely taken into consideration, developers trust in the path they have defined for the users to follow and often forget to prepare for unexpected paths discovered by the human factor, actions that users thought on the fly while making use of the software or application. And it is in the random decisions performed by users that the biggest and more surprising setbacks are found. To reduce the opportunities for this to occur when calculating testing needs manual testing with physical individuals performing random actions is crucial.

Each of these techniques has its own value, each has its own advantage and disadvantage. Using two or three methods of estimation is definitely a more integral approach, especially if they are considered as core procedures and standard practices before releasing.

It is precisely because of the potential of these pitfalls that testing efforts must be integrated along with development as early as possible. Testing efforts must be elastic following the same nature of development, at early stages a small team tests small portions, midway when the stability increases, testing teams can increase with it, automated testing efforts can follow the same growth and complexity of the test teams and development life cycle and finally when the product reaches pre-final or final stages, testing teams are then readjusted to provide real time testing, automated testing and manual testing, an integral testing coverage that has grown and evolved in a parallel path along with development.

Testing estimation can never be final, one of the biggest assumptions in software development is that quality assurance and testing is not fundamental to deliver a successful product. It is always perceived as an after need if there is time, if there is budget, if the product developed seems not stable enough. This assumption affects testing estimations and needs, often drastically affecting the way the final product is perceived by final Users. Testing and QA efforts should walk along the same life cycle as development, even from within the design process, because it can deliver crucial information development needs in order to prepare for that Human factor that often finds the most unexpected bugs within the delivered software

When testing a system that is to be integrated with an existing, in-service / operating system or sun-system, your testing plan will most often be at the mercy of the in-service / operating system interface. Test planning on these situations often gets in trouble when estimates and test plan are “locked in” ignoring the dynamics of this in-service interface (RISK). Test plans need to be dynamic and reviewed frequently to manage and continually mitigate these risks.

Test estimation is also closely linked, if not, highly dependent on the testability of the product. If there isn't the appropriate and relevant stakeholder to test or provide insights into the ability to test, the estimation that's therefore provided will not be relevant. Considering what is being tested, agreeing who will be available and able to test is a fundamental aspect that is often missed. How it's going to be tested plays a huge factor in evaluating the time and expense of the testing. If we are unable to answer these questions, then an informed consideration cannot occur.

Incomplete Requirements or analysis of the sprint or iteration. If my test analysts and engineers are uncovering more than 15% of the requirements it means we need to be more thorough up front. If testing staff is expected to figure out all or most of the requirements and acceptance criteria you need to include that in your estimate or whatever unit of work submitted for an Agile iteration.