testiloquent
Testing-Essentials ▪ Think Like a Tester ▪ Test Strategy ▪ Test Tooling, Automation ▪ Test Analysis and -Design ▪ Performing Tests and Reporting ▪ Appendix
About Test Analysis
The only common testing truth I know of is that all tests can’t be performed. To perform really good tests it is important to know which test cases can be run. To find out which tests that can be run; and which tests that are most important to run in the given context – Edgren
Guided by the test strategy, the test analysis activities examine the test basis, gathering the information necessary to decide how to test each part and aspect of the system to fully achieve the information objectives.
What the analysis involves will in part depend on what the tests in question are to focus on
- the implementation of the system, or on the behaviour of the system
- units of the system in isolation, or the system as a whole
- checking / reviewing project work items (static testing), or running the system under test (dynamic testing)
Test Levels
See Alex Kotloarskyi’s thoughts on test levels and the scope of tests.
Unit Tests
The test analysis at this level may take into consideration:
- The solution design - developers will make solution design decisions before coding, for example using CRC Models
- The available tools. Tooling for static analysis is quite advanced for modern programming languages. In larger IT departments these tools will be integral to the development, integration and deployment processes. Developers working on legacy systems may have limited access to advanced tooling, which will affect the choices they make about what to test and how.
- The requirements specific to the software change that the unit contributes to. such as the acceptance criteria in the user story being worked on, service level agreements or compliance regulations.
- Any quality criteria or risks highlighted in the test strategy.
- Coding guidelines, those laid out in the organisation’s developer handbook, agreed upon at team level or recommended by industry experts such as ‘Uncle’ Bob Martin with his Clean Code.
Developers are very fond of automation, which is understandable - as we established in the previous section, automation can truly be a game-changer. Yet even at the lower testing levels, not every conceivable test ought to be automated.
Read more on that in Brian Marick’s paper When Should a Test Be Automated?
Integration Tests
In broad strokes, the test analysis will likely take the same points into consideration as at the unit testing level. Including higher-level design specifications such as system architecture in the test analysis will be more common at the integration level than at the unit test level. Tooling and relevant quality criteria and risks in focus will also likely differ.
System Tests
These tests can be run once the system has been built, are more likely to treat the software like a black box and focus on behaviour rather than technical implementation details. Thus, the test basis for system tests may be broader in scope than it would be for unit and integration tests.
Test Basis
User Stories
Why the Three-Part User Story Template Works So Well
How Programmers and Testers and Others Should Collaborate on User Stories
Use Cases
https://youtu.be/baY3SaIhfl0
Process
Product development is always a team effort; in fact, it usually involves multiple teams from different disciplines. So the best work systems for development are ones that promote communication and cooperation. – Poppendieck
Your team may be using something called Acceptance Test Driven Development (ATDD), Behaviour Driven Development (BDD) or Specification by Example. Consider yourself lucky. Done well, these processes decrease the effort required for building quality into the product and for effective testing. Required system behaviour is defined in the form of examples, and each example is a test. The more complete the set of examples, the more complete the requirements documentation is.1 The examples keep requirements, tests and the system connected - no outdated test scripts or requirements documentation.
In this article, Liz Keogh recounts how ATDD, BDD and Specification by Example came to be.
Previous: Test Analysis and Design ▪ Next: About Test Design.
-
Gáspár Nagy and Seb Rose. The BDD Books - Discovery (Kindle Locations 161-165) ↩