testiloquent
Testing-Essentials ▪ Think Like a Tester ▪ Test Strategy ▪ Test Tooling, Automation ▪ Test Analysis and -Design ▪ Performing Tests and Reporting ▪ Appendix
Testing in IT Organisations
Testing activities
The testing activities in software development can be loosely grouped into the following categories:
- Initiating and planning - determine what you’re meant to achieve (your information objectives) by when and with what. You may create work items such as the test strategy and test plan.
- Analysis, design - determine how best to achieve the information objectives given the test strategy and time frame. You may create work items such as test charters or feature files and examples.
- Implementation - set up test environments and tools, write test scripts, create test data…
- Performing, monitoring and reporting - run tests, analyze results, report bugs, monitor progress, keep stakeholders informed, adjust test approach as necessary.
SW testing can differ greatly from one organisation to another or even within an organisation from one project to another.
“Traditional” Projects
In larger projects applying a sequential SDLC such as the V-Model, development teams may unit test their system components, while a team or teams of test specialists, led by a test manager are responsible for testing the system as a whole.
As a rule, sequential SDLC projects have hierarchical structures with clearly defined processes, deliverables, roles and responsibilities. The testing process may be taken from the ISO 29119 standard for software testing.
Lean, Agile and DevOps
Agile Testing
While in the V- and W-Models, testing is featured prominently, it is conspicuously missing in the Agile Manifesto. Can we infer that Agile teams don’t test their software?
Not at all! In Quality Assurance in IT we introduced the Agile Manifesto that outlines expectations regarding the quality of the products that Agile teams produce as well as how Agile teams work.
They produce working software, satisfying the customer through early and continuous delivery of valuable software. Their work culture and process is characterized by self-organisation, trust, attention to simplicity and technical excellence, and improvement. Agile testing is an integral part of the development process, helping teams answer the questions: “Will this software work?”, “Will it work well enough to satisfy the customer?”, and “Is this software going to be valuable?”.
Team members will contribute to testing based on their strengths: Developers will likely test changes at the implementation level. Testing of the integrated system will be carried out by whoever on the team has the necessary skills and know-how, be it a product owner, business analyst, test specialist, or a developer.
In larger IT departments multiple teams may be involved in developing a software product or feature. These teams may then collaborate on testing the integrated system.
DevOps Testing
In DevOps teams, there tends to be a strong emphasis on automation to reduce the impact of dependencies and increase efficiency.
As in Agile teams, the responsibility for quality assurance measures lies with the team, however DevOps teams are responsible for the system throughout its lifecycle.
Test Levels
Testing levels are made explicit in the V-Model SDLC diagram, but a layer model of the system under test is not exclusive to sequential SDLC projects, but widely used in all types of software development projects.
Tests at different levels are suited to different information objectives.
- Testing at lower test levels (unit and integration levels) can be well suited to answer the question “Will the software work at all?”1, and can provide preliminary information about how well it will work, for example by identifying slow database queries.
- Testing at higher test levels (integration and system levels) can provide further answers to the questions “Does the software work well?” and “Is it usable?”.
While we will discuss three test levels called unit, integration and system, upon starting work at a new organisation, you may well discover that they use differnt names for their test levels. Also possible is that their model has more granular test levels defined, with perhaps ‘Component Tests’ between Unit and Integration Tests, or ‘System Integration Tests’ above System Tests.
There are multiple forms that test level models can take. It is common for them to be represented in a triangle, usually called a Test Pyramid, with lower test levels taking up more space to illustrate that the bulk of tests should be closer to the implementation.
Read about the test automation pyramid.
Unit Tests
What counts as a unit may differ from one technology stack to another. In Java programming for instance, a unit is usually understood to be a short piece of code with narrow capabilities. In COBOL programming on the other hand - a language in which many business-critical legacy systems are written - a unit might be an entire module. In both cases, tests at this level are close to the code and of limited scope.
As a rule, unit testing is up to developers, who will run automated dynamic and static tests (also called static analysis or code analysis) whenever changes are checked in, to check the implementation and verify that the unit being tested behaves as the developer intended.
Dynamic Unit Testing
Good unit testing is the foundation on which effective testing at higher levels rests. The advantages of unit tests, as compared to tests performed at higher levels are:
- Unit tests are less expensive to create and run.
- Analysing the cause of failed unit tests is usually simpler than failures at higher levels.
- Unit tests provide faster feedback to the development team.
Industry experts recommend test-driven design (TDD), that is, writing a test before writing code that passes the test. TDD effectively prevents the kinds of bugs that unit tests can detect from being introduced into the code. However, it is more common for developers to write their unit tests after the production code has been written.
Unit tests are often designed to use internal details such as the code and the data (white-box / glass-box tests), however it is possible for a black box approach to be taken when designing tests to check the behaviour of a unit.
Watch a unit test being created in this video: Unit Tests in Python.
Static Code Analysis
Automated Code Analysis
For modern programming languages, advanced tooling for static analysis increase the efficiency of development, integration and deployment. The tooling available to developers working on legacy systems may lack helpful features or be difficult to use, which will affect the choices they make about what to test and how.
Code Reviews
Most development teams have a process in place for code reviews, in which the team members check each others’ code, including the unit test code.
“Your” developers may be open to you participating in code reviews. Depending on the organisation and team culture, you might even be expected to review code. Understanding how the system under test works can lead to new test ideas, and contributing to code reviews is an excellent opportunity to learn more about the system under test.
Integration Tests
These tests check the interfaces between individual parts of a system and how the parts interact.
Knowledge of the architecture of the system is usually necessary to design integration tests, but the resulting test scripts don’t necessarily need to use internal technical details - integration tests can treat the software like a black box.
Static tests at this level include reviews of system designs.
System Tests
Dynamic tests at this level can be run once the system has been built. They concern themselves with the behaviour of the system as a whole, and to this end, will likely treat the software like a black box.
Previous - Quality Assurance in IT ▪ Next - Think Like A Tester.