Think risk-based to prioritize the tests

You can never test everything, therefore you need to think about what business risks exist in your development project. With this as a starting point, you can plan the order in which you will test, how you will minimize the risks around tests and how the tests will be prioritized, so that you test the most critical first. Your test strategy will then have the same business priority that governs the entire development project. In this blog post, I describe how you should think when it comes to risk-based testing. 

There is a business perspective to how important it is to test. The industry your system is developed in is also important. If your project is within, for example, banking, insurance, medical technology or government agencies, the company has a reputation that it should work correctly. Even if the mistake is just a misspelling, it can lower confidence in the entire company. In other cases, it may be most important for the company you work to get a product to market before everyone else, then it may be possible to accept that there are some bugs in the first version of the product. In all projects there is usually a priority between time, cost and quality. If you prioritize quality, it will affect time and cost. If time is most important, cost and quality will be affected. 

time, quality and cost

In other words, the time, cost and quality factors are also important parameters for your test planning. There is actually a possibility that you are testing “too much” in relation to the overall priority of the project. This is important so that the testing effort takes place with the same goal against which the entire project is to be measured. When you plan your tests, you produce your test cases, test scenarios or the prerequisites for exploratory tests and get a list of everything to be tested. But not all tests are equally important, there are always functions in the system that are run many times, while other functions are run less often. There are also other factors that control whether a function in the system requires more or less testing. How complex a function is in terms of requirements and development has a lot of weight in your prioritization of the tests. 

A risk-based test is based on a risk analysis usually based on what is to be tested within a particular test level, for example system test or within a sprint. The type of risk analysis you choose is not very important, the important thing is that you take into account the probability and consistency of aspects selected by the project such as business critical, implementation complex, data complex and more. Also consider the time, cost and quality factors mentioned above. 

How to carry out a risk-based test analysis

A risk-based test analysis can be carried out in different ways, this is a proposal that shows how you can do it. This may not be right for you and your project, but we hope you can get some good tips that you can use. 

1. Appoint a person as risk manager, who will run the risk-based test analysis – preferably the test leader.

2. Risks must, first of all, be identified within the scope of the assignment, which can be done in a classic way with the help of “yellow notes”. Or why not use Mind Mapping, where you write down risks – high and low. This is best done in a group, in a common room for instance, so that you can be inspired by each other, become more creative and think outside the box.  

3. When the collection of risks is completed, the risks must be grouped, which is usually done in functions or areas. But there may be other ways of grouping the risks that suit the mission better. The grouping is done by the risk manager. 

collection of risks

4. The risk manager calls a new risk analysis meeting where he presents the risks and their grouping. Now the participants can comment and add new risks, if necessary the risks can be broken down in more detail. After that meeting, the map of risks must be ready and the prerequisites must exist to carry out a risk analysis where the probability and consequence are evaluated.

5.The risk manager calls a new workshop where the risk analysis itself is carried out. Examples of participants are test leader, experienced tester, customer, architect, Project manager, product owner/system manager, experienced developer.

6. Probability and consequence must be weighed up for each risk. Start by going through the probability that the risk will occur, 1-4 and enter it into the spreadsheet. 

Probability 

When it comes to probability, we take into account various factors that affect the probability that the risk will occur. 

Probability of risks

Continue to weigh the consequences for each risk. Hide the Probability column so that it does not affect the weighting of the consequence. 

Consequence 

Here it is important to identify all relevant stakeholders and think about whether some of these are more important than others and, if so, how they should be weighed. 

Consequences of risks

7. Multiply the Probability by the Consequence and you get a risk value. Distribute the risks according to their risk value:

  • 1-4 = low 
  • 5-9 = between 
  • 10-16 = high 
Risk Probability

8. Decide how to eliminate the risks:

  • Whether to eliminate all risks at once by starting with those with the highest risk value and eliminating downwards. 
  • Whether to start from the respective function/area and then eliminate the risks. 
  • Whether to start from deliveries/sprints. 
  • Whether to write test cases/scenarios at all/provide conditions for exploratory tests for those risks with a low risk value? 

9. Start working with the risks that have the highest risk value:

  • Check if the risk can be linked to a requirement or the equivalent, note it in the column Mapped Requirement. 
  • Identify one or more test cases/scenarios/conditions for exploratory tests or equivalent that check that the risk is eliminated. Write them at the header level to get an overall view of how big the test work will be.
highest risk value

10. Then it is time to start writing test cases/scenarios/providing conditions for exploratory tests at a detailed level.

  • Appoint responsible persons for each test case/scenario/condition for exploratory tests.
  • First write at header level to get a feel for how many tests are to be carried out. It is suggested to be done in a test tool, alternatively continue to expand the excel sheet, which will be very complex. 
  • Write the test cases/scenarios/prerequisite for exploratory tests at detail level. 
  • Consider whether the test case/scenario should be executed automatically or manually. 
  • The test cases/scenarios should be reviewed before they are executed. 
  • If possible, in which order the test cases/scenarios/exploratory tests should be performed. As a suggestion, the tests with a high risk value should be tested first, but there may be basic functionality that you want to ensure is working before starting the other tests.

 11. If you want full requirements coverage, in parallel with the tests being detailed, you can check how well the requirements are tested. In point 9, risks, requirements and tests have been mapped, now you can check if there are requirements that do not have tests defined. If there are requirements that are not covered by test cases/scenarios/exploratory tests, note them and enter them, preferably under a new tab in the Excel sheet. 

12. Perform steps 3-10 for the requirements that were not covered by the risk analysis. Manage the requirements in the same way as the risk, i.e. produce a risk value, which helps you plan in which order the tests should be produced and carried out. 

13. Now you have a system that will be tested with full requirements coverage and you have eliminated all high risks by having test cases/scenarios/exploratory tests that will ensure that all important functionality is tested. 

14. When it is time for the next delivery/sprint, it is useful to follow up on previous risk analysis and ask yourself:

  • Can the probability/consequence of the risk be changed? Have we eliminated or partially eliminated the risk with the tests developed/performed?
  • Have there been any new risks or requirements that need to be treated in the corresponding manner described above.

15. When planning the upcoming delivery/sprint, do the same as described above, but regression tests must also be identified here. A tip is then to select test cases/scenarios/exploratory tests where deviations have been found and corrected and select test cases/scenarios/exploratory tests with a high risk value.

—

Hope this gives you some help on the trot, all tests are different and you need to improvise so that it suits the organization, test level or if it is in sprints you are working with. 

About the author

QESTIT Team

Wish to know more about testing?