Making Your Testing Effective

I’m sure most of us will have worked on projects where without having to do any particularly extensive test scenarios or having hardly dived into exploring the product, you find things that break that would be immediately obvious to the customer. This kind of project depresses me (even though it is fun to find defects), I like nothing more than a project where I struggle to find defects and all user scenarios work correctly, as this means (generally) that the code has been well thought out, it usually also means I have been able to work alongside the developer and ensure that quality has been built into the code and of course it may also mean that the customer has been listened to and the requirements have been developed against. As mentioned in a previous post, for me, testing starts as soon as I have started thinking about a project, so if we have got to a point where user scenarios are being met and minimal defects are being found, this would suggest I have done a reasonable job of asking the right questions and verifying that all the features work correctly through whatever method of test execution I have followed. But if I did have to raise a lot of defects, and the user scenarios didn’t work, does that mean my testing wasn’t effective? Of course it was, if it hadn’t been effective, I wouldn’t have found the issues and proved that the level of quality needed for release wasn’t there. Testing can be effective in both scenarios, it’s just about preparing your testing in the right way, so you can have confidence in what you are doing, regardless of the standard of the product under test.

So how would I define ‘effective’ testing? I would suggest these ideas:

1. Know The Product – Read any documentation you can. Ask any questions which will help you understanding (however stupid they may seem, if you need to know the answer, then how stupid can it be?). Review code and unit tests to understand the flow of the system. Spend time exploring the paths through the product so that you are familiar with the expected routes from end-to-end. Having full confidence in the product will ensure you know what to test.
2. Understand the Risk – Fathom out the areas of the product which pose the most risk to testing and ensure that these are prioritized before you start writing any scenarios. Understanding how to mitigate the risk for each of these will enable you to focus on testing the areas fully.
3. Document Your Plan of Action – Now that you feel confident about the product, find a way which works for you to document your planned testing. This may be a mindmap, a word document or even scribbled on the back of a beer mat ;). What is important is that you have some way of being able to look at what you have thought to test and identify if there are gaps.
4. Review with Peers and Stakeholders – Now maybe that beer mat wasn’t the best idea. It would be easy to hand your test plan over to someone else to look at, them to skim through it and hand it back to you without comment. Reviewing for me, would ideally mean walking through the test plan as a discussion with team members (dev & testers) and also with the people who provided the requirements. Talking through the plan with others will highlight how confident you are with what you are going to test, you may find additional scenarios to cover and also maybe get some suggestions on how to improve your approach.
5. Be Prepared to Evolve Your Testing – There’s a good chance your original plan will not be exactly what you end up completing when executing scenarios. There will be additional options or configurations which may not have been obvious before. Don’t be afraid to divert from the plan, it should never be a hard and fast commitment, it should be seen more of a ‘Plan of Intent’.
6. Review Results – When you have completed your test plan, go through the results again, ensure you understand the outcomes. Discuss with another team member who may highlight something that you hadn’t initially noticed.

No two projects may be the same, and there may not be one particular approach to Test Scenarios which can be used for all projects, but following these suggestions will hopefully ensure you have confidence in your testing and will give you a definite view of the quality of the product.

The ultimate goal is to ensure that your testing is doing it’s job, it’s finding defects if there are defects to find and it is proving the good quality of the product so that the team can have a distinct understanding of when it is good to release.