There have been several articles mentioning answer to this problem, is my testing enough? Are we ready to ship the Software?
Are my reports enough to prove that we are at a Signoff state?
Software testing is one niche where probably there can never be enough of testing. However, given the budget and time constraints, you cannot avoid the all critical question, ‘How much testing is enough?” There cannot be any precise answer to it as it actually depends on case to case basis after taking all considerations.
General tips for deciding to stop the software testing
Though there is no hard and fast rule to put an end to software testing, the most important considerations to put an end to it are:
●Deadlines- There is a constant pressure to develop the software and release them at given deadlines. Often, provided no major hiccups were reported, the testing should be stopped according to deadlines. This is the easiest and simplest way to decide the end.
●Test cases– When you have completed a number of test cases successfully and inferred as certain success percentage. Only open challenge and question is that were the test cases carefully written and covered everything?
●Budget depleted- Allocating budget for software testing can be an uphill for a company especially a start-up company. If the allocated budget has depleted, then the software can be released if successful results are obtained. If the test results are not satisfactory; then you might need to shelve the software till you procure more budgets.
●Coverage of code, requirements or functionality reaches the specified point- This is the bear set even before the testing begins. Once this is achieved, you know that you can stop the testing.
●Bug rate falls below specified level- No software can be free of bugs. However, if during testing, its percentage falls below the threshold level, it is time to stop testing and release the software.
●Beta or alpha testing period ends- If the testing period is over, you can reasonably stop the testing.
Other Guiding Principles
Apart from the general guidelines, there are several other situations which determine the extent and duration of software testing. After conducting series of tests you might be satisfied with the quality of codes or the bug percentage but the reality might be a completely different picture. The quality, bugs percentage etc. could be pointed in just the opposition direction. If you release the software in this state, it may prove expensive and detrimental to your business. You can control it by evaluating the possibility of finding more defects. This is inferred from the analysis of the results obtained so far. The analysis is important because
●A newfound defect is a strong indication to continue testing as often they lead to other defects.
●If you found defects with a small portion of overall functionality, you should continue the testing.
●If multiple testing of the functionality of the software is not showcasing any defects, then it is the right time to stop testing the software.
●Before putting a stop to software testing, you need to ensure that its significant features are mostly or entirely tested for satisfactory performance.
●One should always rely on comprehensive tests and results and not based their decision on a false confidence veneer. It is not uncommon that many developers completely skip on tests stating that as long as the codes comply; it has reached the acceptance testing benchmark.
Success comes from identifying the risks early. It is a good indicator of when to stop software testing. The risk factors will determine your level of testing. If in various testing like Unit testing, System testing, Regression testing etc., you are getting positive results, then you can stop testing.
We can easily surmise that you can understand it is time to stop software testing when you are confident of finding any additional defects and when various tests give you a high degree of confidence that the software is ready to be released.
