testiloquent
Testing-Essentials ▪ Think Like a Tester ▪ Test Strategy ▪ Test Tooling, Automation ▪ Test Analysis and -Design ▪ Performing Tests and Reporting ▪ Appendix
Test Strategy
About Strategy
A firm creates a sustainable competitive advantage over its rivals by “deliberately choosing a different set of activities to deliver unique value.”
Strategy therefore requires making explicit choices - to do some things and not others - and building a business around those choices. – Lafley, Martin.
Martin identifies the following elements of a strategy:
- A winning aspiration
- Where to play
- How to win
- Capabilities
- Management systems
A test strategy covers the same bases, from the perspective of a testing effort. The test strategy is the set of ideas that guide your test design, and the foundation upon which you can plan, deliver and assess the success of your test project.
Why a Test Strategy?
Your ability to make appropriate choices about your testing activities depends on this part of the test process.
No two organisations or projects are identical, so no one test strategy is an ideal fit for every situation.
Test Strategies Small and Large
Organisation- or project-level test strategies
Test strategies come in different shapes and sizes, addressing testing at scales ranging from the work of a single tester to testing in an entire organisation. The latter may be a large formal document, upon which formal test plan documents for each new testing project are based. These documents may be drawn up at considerable expense, only to soon be out of sync with the realities of testing in the ongoing projects.
Test strategies for individuals and teams
Strategy is an iterative process in which all of the moving parts influence one another and must be taken into account together. – Lafley, Martin.
The best test strategy is one that contains actionable and context-relevant information that helps you achieve your testing mission.
A test strategy at the project or organisational level will be too unwieldy and generic for individual testers preparing an iteration. It’s created for a different context. However, when you’re developing a strategy for the next iteration, it’s advisable to use high-level test strategies as a guideline.
Although your organisation-level test strategy may be a lengthy formal document, your team’s test strategy need not be. Light-weight documentation has the benefit of being easy to understand, and thus especially useful for communicating ideas. D-Day was a massive endeavour by any standards, but General Montgomery captured the strategy on this 1-pager.
The Heuristic Test Strategy Model
We will focus on developing a test strategy for the resources you can control the most - your own work and time, applying patterns from James Bach’s Heuristic Test Strategy Model (HTSM).
The Heuristic Test Strategy Model is a set of patterns for designing and choosing tests to perform. The immediate purpose of this model is to remind testers of what to think about during that process – James Bach
A test strategy developed with the HTSM may cover all of the bases in the strategy model described in Lafley and Martin’s Playing to Win:
Key information in a test strategy
The test mission - your winning aspiration
The strategy is grounded in a defined purpose, centered on customers. The winning aspiration sets the frame for the rest of the strategy.
- In the heuristic test strategy model, this is the first node under Project Environment: the “Mission”.
- You will need to work with your customer/s to understand and clarify what the testing is to achieve.
- The mission is not fixed in stone. As the project progresses, it will make sense to pursue different aims. Moreover, you and the other project team members will learn more about the project context and the system under test over time, so you’ll want to refine the mission and adjust the strategy accordingly.
The test level and quality characteristics - where to play
The next element of strategy is defining a “playing field” on which the aspiration can be delivered.
Just as entrepreneurs know that no company can be all things to all people and still win, all testers know that complete testing isn’t realistic. You can’t test everything, so you figure out which where-to-play choices will best enable you to fulfill your testing mission, given the project circumstances.
- The heuristic test strategy model provides ideas for many different “playing fields”.
Use them to define the focus and scope of the testing which will allow you to fulfill your mission.
Test types and methods - how to win
Where-to-play and how-to-win choices must be considered together, because no how-to-win is perfect, or even appropriate, for all where-to-play choices.
- In the heuristic test strategy model, the “how to win” question would center on maximizing the value of testing and aligning the test activities, for example through:
- The choice of test methods
- Choosing how to implement tests (for example, what to automate, what not to automate)
- Choosing what reports to create, for whom, when / how often and in what format.
Capabilities
Given the choices made above, the various activities needed to fulfill the mission can be identified.
- The testers will require certain resources and core skills to carry out these activities.
- Perhaps some changes to the code will need to be made to enhance testability, or tools may need to be made available to make the how-to-win decisions feasible.
The Capabilities topic is addressed in various nodes of the Project Environment branch of the heuristic test strategy model.
Management systems
Given the choices made above, the management systems would include infrastructure or support needed to carry out the activities (1), as well as any systems needed to evaluate the effectiveness of the strategy (2).
- The list of management systems might for example include an IT operations process to configure test environments, seed the database, and tear down test environments.
- A test mission clearly understood by all involved is a prerequisite for determining which indicators to use for monitoring our testing, evaluating its effectiveness and knowing when the testing can stop.
Furthermore, we want to gather information about the testing process to identify potential for optimization.
A Strategy is Not a Plan. A Plan is Not a Strategy.
Examples of light-weight test strategy documents
The following examples from the context-driven testing community are very specific to the way one person works / to a particular project environment. These testers would likely have started with ideas from the heuristic test strategy model and changed the headers and structure, added bits and left lots out.
