Testing-Essentials ▪ Think Like a Tester ▪ Test Strategy ▪ Test Tooling, Automation ▪ Test Analysis and -Design ▪ Performing Tests and Reporting ▪ Appendix
Dynamic tests put the software product (or individual components) through its paces, while static tests analyse artefacts such as system designs, documentation or code.
Tests that use internal details of the application are classified as white- or glass-box tests; those that do not use the internal details are classified as black-box tests.
Some techniques can be used in either a black-box or white-box way, some are more likely to be used in a white- or glass box way, and other techniques are more likely to be used in a black-box way.
Test techniques can be used both for exploratory and scripted testing.
In test design, techniques are recipes for gathering information about software. There are many test techniques to choose from, each of them with its own strengths.
The project situation and information objectives can change over time, and the choice of test techniques may need to change accordingly.
For an overview of commonly used test design techniques, watch this lecture from Rob Sabourin.
Of Testing Tours and Dashboards
Tours are useful for getting to know a system, focusing on one aspect at a time.
The result is an inventory of the things we are interested in, and quality-related information about those things that guide further testing.
Tours lend themselves well to working in pairs. They are a great way of introducing new team members to the system - and new team members, looking at the system with fresh eyes, may be able to contribute new test ideas or notice anomalies that experienced team members have missed. The amount of time you take for the tour will depend on the context; it can be carried out in more than one session if necessary.
Depending on the focus of your tour, you may have system utilities or other tools available to you that can help you gather more information or to work more efficiently.
Variable Tours
A variable tour concentrates on examining anything that can be changed in the system.
Data Tours
What values that can be fed into those variables? Are there valid ranges? Boundary values? Where does the data come from (calculation, user input, other programs…)?
Feature Tours
A feature tour gathers information about the system’s capabilities.
Further information about the system’s functions and capabilities can be gathered from specifications, user manuals, help files, claims made by sales…
File Tours
A file tour works through the system’s directory structure and examines the files that the software uses, or that are a part of the system.
Error Message Tours
Coverage-driven techniques organise testing around an inventory of ‘things’ to test, and a coverage level goal.
Code coverage - Static code analysis tools for modern technology stacks generally support control flow-, branch- and decision coverage checks.
Functional (-requirements) coverage - A feature tour can produce an inventory of functions to cover, as can functional requirements specification documentation.
State transition or state model testing covers an inventory of operational modes and transitions represented in a model of the system.
Decision table testing covers an inventory of decisions, such as business rules.
Domain testing, a technique that expands on Equivalence Class Partitioning and Boundary Value Analysis covers variables and the values they work with.
User story testing
Use cases provide step-by-step descriptions of an application’s features. They may created as part of an application’s requirements specification documentation, to describe how the system is expected to behave. The steps are commonly presented in a use case diagram.
The tests for the individual steps in the use case will be designed using the techniques most appropriate for checking the behaviour of the system under test against the claims in the use case.
People who understand their own business and wonder how well your program meets their needs will often test with scenarios. A scenario is a story about how people will use software in a real-life situation. The test is designed for realism, not power. Failure indicates that something important is wrong with the software. – Kaner
Scenarios are similar to use cases in that they detail steps that could be taken through a system. They won’t be part of requirements documentation, but may rather be a product of risk analysis, or created by testers. While use cases document expected functionality, scenario tests describe and explore realistic situations that the product may not be prepared to deal with, in order to see if - should the situation arise - stakeholders might experience difficulties with or because of the product.
The information you’ve gathered during the test analysis, the decisions made in your test strategy and test plan, and the test design choices you made now all come together.
Before tests are performed, the test environment needs to be configured. Any test scripts required will have to be written or updated. In a sequential SDLC project, it’s common to arrange the test scripts into test procedures, and set up test suites in order for testing to be performed as planned during the test phase.
Previous: About Test Analysis. ▪ Next: Performing Tests and Reporting.