This article describes how research and technology risks could be classified. This risk classification is the basis for the attribution of appropriate and proportionate legal obligations in the prototype regulation presented in the following blogpost.

I. The purpose of risk classification

1. Not all research projects or technology are dangerous. The vast majority are not. But some are, and they are so, to differing degrees. This should be reflected by regulation. It would be disproportionate to burden low-risk research projects or technologies with heavy requirements (e.g. on risk control), whilst it might be fully justified to burden high-risk research projects or technologies.

2. In theory it would be conceivable to have an endlessly fine-tuned risk scale and to develop appropriate requirements in regulation. In practice it is not. To reflect different risks, the best possible approach is to create risk classes and to establish requirements which are proportionate to the risk class.

3. Based on the previous blogposts on research risks and on technology risks, the next blogpost in this series will present a prototype regulation. To prepare this final and fourth blogpost, this blogpost presents the missing preparatory element, the risk classification. The risk classes shall be cast in such a way that, together with the scaled obligations under each risk class, research and technologies shall not be disproportionately hindered or even just slowed down, but still efficiently controlled so as to reduce the high risks linked to them.

II. Basic questions of risk classification

4. When regulators evaluate research undertakings and technologies, they might apply two different goals:

– Reducing harms or risks;

– Risk-benefit optimisation.

5. The problem with risk-benefit optimisation is that the benefits occur to other persons than the risks, as do financial benefits and costs. Furthermore, it is even more difficult to evaluate benefit than to evaluate risks. Hence it is much easier not to take account of benefit at the level of procedures to be applied to research and technology risks. The benefit will thus not be reflected in our risk classification. However, it is fully rational to take account of potential benefit at the level of decision making, and be it in view of providing a derogation to undertake research or apply technologies despite risks which, otherwise, would be deemed to be too high to be accepted. We will therefore integrate a special derogation clause in the prototype regulation developed in the next and final article of this series.

3. When pursuing the goal of reducing harms or risks, the question arises how the harm or the risk shall be evaluated or which definition of harm or risk shall be applied. In the international standards on risk reduction and other literature, we have identified the following most frequently used parameters for evaluating harm or risk or for defining harm or risk:

Severity: Seriousness of harm

Scope: Number of persons affected by harm

Likelihood: probability of the harm occurring

Lasting of harm: unless taken into consideration by Severity or Scope

We will base our further modeling on these four parameters.

The expressions harm and risk are sometimes used synonymously. Mostly, however, risk is defined or implicitly understood as harm multiplied by probability. We use here the second definition. We use the term “risk” also in the following way:

Death risk = a likelihood of death >0

Risk = likelihood of harm >0

4. Often it is not sure how the future will evolve. Where different scenarios are conceivable, one can evaluate them separately and attribute likelihoods to them. This is one way to cope with scientific uncertainty. Other ways consist of factoring in safety margins or safety factors. Whilst this is all reasonable, we will not enter this debate here, but focus on the modeling as such.

5. Before coming to the models as such, please read carefully a few preliminary remarks which apply to all of them:

– Indirect risks need to be taken into account. E.g., a computer virus might indirectly cause the dis-functioning of an electricity grid, thus affect a hospital and cause deaths by disrupting life-preserving machines.

– In case of multiple harms for the same victims, such as suffering followed by death, it is appropriate to evaluate both and to classify risks separately.

– Regulators who give a value to sentient beings other than humans or to nature, can introduce additional rows to cover harms affecting them.

– A particular risk assessment is needed when research or technologies might lead to the extinction of mankind. The extinction of mankind would stop the potential not only of billions, but trillions or even quadrillions of humans who could live over the next millions of years. This is not reflected by any of the models below. For further reflection on this aspect, we recommend the writings of the existential risk pioneer Nick Bostrom and in particular his article Existential Risk Prevention as Global Priority.

6. Furthermore, before coming to the models as such, we provide here a limited preview on the obligations that we will recommend to link to each of the five risk classes:

Class I: Self-certification without quality management system followed by registration in a public database

Class II: Self-certification with quality management system followed by registration in a public database

Class III: Certification of the quality management system by an entrusted body, self-certification for other aspects, both followed by registration in a public database

Class IV: Certification of the quality management system by an entrusted body + State authorisation procedure with consultation of national experts.

Class V: Certification of the quality management system by an entrusted body + State authorisation procedure with consultation of international experts.

III. A few key models of risk classification

7. Evidently, it is very cumbersome for regulators to develop a multi-parametric risk classification system based on the factors identified. In order to facilitate the task, we present here a few models which regulators can take as a basis.

8. To do so, we come first back to the four key parameters presented above:

Severity: Seriousness of harm (which can be roughly represented by the types of harm: death / health loss / financial loss)

Scope: Number of persons affected by harm

Lasting: duration of harm

Likelihood: probability of harm

Risk = likelihood of harm >0

9. Outline of MODEL A:

Class I: No death risk (0 likelihood of death)

Class II: Death risk for less than 10 persons (>0 likelihood of death for less than 10 persons)

Class III: Death risk for 10 to 1000 persons

Class IV: Death risk for 1000 to 1.000.000 persons

Class V: Death risk for more than 1.000.000 persons

10. The most simple model for risk classification focuses on the risk of death and takes just the parameter “scope” into further consideration. The death risk indirectly also mirrors to some extent the risk of being hurt or losing health. The most simple model avoids to a large extent the complicated and debatable assessment of the likelihood that will play a role in the following models.

11. Evidently, the thresholds (10, 1.000, 1.000.000) are somehow arbitrary and thus can be modified.

12. Outline of MODEL B

	Class I	Class II	Class III	Class IV	Class V
Death risk for	None	Less than 10 persons	10 to 1000 persons	1000 to 1.000.000 persons	More than 1.000.000 persons
Health risk / Risk of injury for	Less than 10 persons	10 to 1000 persons	1000 to 1.000.000 persons	1.000.000 to 1.000.000.000 persons	More than 1.000.000.000 persons
Risk of income loss / wealth loss for	Less than 1000 persons	1000 to 1.000.000 persons	1.000.000 to 1.000.000.000 persons	More than 1.000.000.000 persons	Mankind

13. Model B expands the simple Model A by additional types of harm. Thereby, we shift from a one-dimensional to a two-dimensional structure. But the severity of the harms are not yet taken into account. To do so, we turn to Model C.

14. Outline of MODEL C

	Class I	Class II	Class III	Class IV	Class V
Death risk for	None	Less than 10 persons	10 to 1000 persons	1000 to 1.000.000 persons	More than 1.000.000 persons
Risk of severe health damage or injury for	Less than 10 persons	10 to 1000 persons	1000 to 1.000.000 persons	1.000.000 to 1.000.000.000 persons	More than 1.000.000.000 persons
Risk of medium severe health damage or injury; risk of important loss of income or wealth for	10 to 1000 persons	1000 to 1.000.000 persons	1.000.000 to 1.000.000.000 persons	More than 1.000.000.000 persons	Mankind
Risk of light health damage or injury; risk of medium important loss of income or wealth for	1000 to 1.000.000 persons	1.000.000 to 1.000.000.000 persons	More than 1.000.000.000 persons	Mankind	–
Risk of not so important loss of income or wealth for	1.000.000 to 1.000.000.000 persons	More than 1.000.000.000 persons	Mankind	–	–

15. Here we see in the left column a modification when compared with Model B. Instead of referring just to types of losses as indicator for severity, we add qualitative descriptions: “important”, “medium important”, “not so important”. Evidently, the qualification is prone to interpretation disputes, as always when we refer to verbal description. This is a downside of this model. Furthermore, the increased complexity might be regarded as disadvantage as such. However, we are still far away from the upper end of conceivable complexity. We will go only one steps ahead with Model D. Model D integrates the parameter of likelihood.

16. Outline of MODEL D

	Class I	Class II	Class III	Class IV	Class V
High or medium death risk for	None	Less than 10 persons	10 to 1000 persons	1000 to 1.000.000 persons	More than 1.000.000 persons
Low death risk; high risk of health damage or injury for	Less than 10 persons	10 to 1000 persons	1000 to 1.000.000 persons	1.000.000 to 1.000.000.000 persons	More than 1.000.000.000 persons
Medium risk of health damage or injury; high risk of loss of income or wealth for	10 to 1000 persons	1000 to 1.000.000 persons	1.000.000 to 1.000.000.000 persons	More than 1.000.000.000 persons	Mankind
Low risk of health damage or injury; medium risk of loss of income or wealth for	1000 to 1.000.000 persons	1.000.000 to 1.000.000.000 persons	More than 1.000.000.000 persons	Mankind	–
Low risk of loss of income or wealth for	1.000.000 to 1.000.000.000 persons	More than 1.000.000.000 persons	Mankind	–	–

17. Model D integrates the parameter of likelihood without losing the parameter of severity if risk is understood in the following way:

Risk = Severity x Likelihood

We recommend, for the context of this Model D, to understand risk in this way so as to make Model D more comprehensive.

18. Furthermore, if we so wish, we can interpret severity in such a way that “duration” is taken into account. Duration is an important factor above all when assessing health damage or injury.

19. So far, we have integrated one parameter after the other. But what can regulators do, when they wish to discard the basic parameter of Model A to D, the scope? They can simplify the Models B, C or D by eliminating the parameter scope and placing the entries of the left column directly under the heading Class V, Class IV, Class III, Class II and Class I. The chart for the simplified Model D (= Model E) would look as follows:

20. Outline of MODEL E (= Model D simplified)

IV. Results of Models A to E

21. The Models A to E are abstract. They must be applied to the various research and technology risks. We do not claim that we dispose of a comprehensive list of research and technology risks. By the following, we just wish to illustrate how various research and technology risks could be classified under the Models A to E. The risks are mostly identical, but in some cases the readers will observe deviations.

Research undertaking / technology application	Model A	Model B	Model C	Model D	Model E
Setting free or other uncontrolled dissemination of organisms falling under the definition of biological arms	Class V	Class V	Class V	Class V1	Class V2
Setting free or other uncontrolled dissemination of genetically modified or artificially created viruses, bacteria or fungi	Class V	Class V	Class V	Class V3	Class V
Genetical modification or artificial creation of viruses, bacteria or fungi in a controlled environment4	Class V	Class V	Class V	Class V5	Class V
Setting free or other uncontrolled dissemination of genetically modified or artificially created plants or animals	Class V6	Class V7	Class V8	Class IV9	Class IV
Genetical modification or artificial creation of plants or animals in a controlled environment10	Class V	Class V	Class V	Class IV	Class IV
Research on or creation of IT viruses11	Class II	Class III12	Class III	Class II13	Class II
Release of artificially intelligent software or hardware which is self-replicant or which has the potential to control an undetermined number of other software or hardware	Class IV	Class V14	Class IV15	Class IV	Class V
Research on or creation of artificially intelligent software or hardware which is self-replicant or which has the potential to control an undetermined number of other software or hardware in a controlled environment	Class IV	Class V16	Class IV17	Class IV18	Class IV
Research on or creation of artificially intelligent weapons	Class II	Class II	Class II	Class II	Class II
Changing the ground below the surface of the earth for a space larger than 1.000.000 cubic meters, but not more than 1.000.000.000 cubic meters	Class II	Class II	Class II	Class II	Class II
Changing the ground below the surface of the earth for a space larger than 1.000.000.000 cubic meters (= 1 cube km)	Class III	Class III	Class III	Class III	Class IV
Changing the ground below the surface of the earth for a space larger than 1.000.000.000.000 cubic meters (= 1000 cubic km)	Class IV	Class IV	Class IV	Class IV	Class V
Establishment of a nuclear installation which can explode	Class IV	Class IV	Class IV	Class IV	Class IV
Establishment of a nuclear installation which can otherwise release nuclear material	Class III	Class III	Class III	Class III	Class III
Operations which might change the weather or the climate in a zone larger than 100 square kilometers (e.g. 10 x 10 km)	Class II	Class II	Class II	Class II	Class III
Operations which might change the weather or the climate in a zone larger than 10.000 square kilometers (e.g. 100 x 100 km)19	Class III	Class III	Class III	Class III	Class IV
Operations which might change the weather or the climate in a zone larger than 1.000.000 square kilometers (e.g. 1000 x 1000 km)20	Class IV	Class V21	Class IV22	Class IV	Class V
Operations which might change the weather or the climate in a zone larger than 100.000.000 square kilometers (e.g. 10.000 x 10.000 km)	Class V	Class V	Class V	Class V	Class V
Operations which might change the weather or the climate globally23	Class V	Class V	Class V	Class V	Class V

22. In case of doubt, research undertakings and technologies were attributed to the highest debatable risk class, applying the precautionary principle. For some research undertakings and technologies, it should be possible to determine sub-groups for which a lower risk-class could be assigned, e.g. by lower level regulatory acts to be adopted by the administration. Therefore, we recommend foreseeing empowerments for these lower level regulatory acts to be adopted.

23. Applied to the research undertakings and technologies listed above, the risk for health and of injury did not change the outcome in the previous chart. Thus it seems that the risk of death is a very good indicator for the risk for health and of injury. Still we are not sure that other assessors will necessarily come to the same result. Furthermore, other research undertakings or technologies might well trigger a risk for health or of injury without triggering any risk of death. Therefore, we kept health and injury in Models B to E.

V. Comparing the Models in light of the outcomes

24. The fact that the risk for health and of injury did not change the outcome lets us suppose that the risk of death is a strong indicator for the risk for health and of injury. More surprisingly, the risk of death is also strongly correlating with the risk of loss of income or of wealth. In but a few cases the risk of loss of income or of wealth led to the assignment of a higher class than the risk of death. These two facts might console those regulators who would like to establish a risk classification model based on various risks, but who cannot do so because of technical or political obstacles. Already Model A which is exclusively based on risk of death as risk class assignment criterion does quite well even when measured against “the ideal” complex Model D.

25. Inversely the findings described in the previous paragraph permit regulators who are primarily interested in simplicity to discard other criteria than the risk of death because the sidelining of other criteria leads only to a few unjustified “misses”. The “misses” occur with regard to loss of income or of wealth only.

26. Model E is another quite simple model. The results of Model E differ however more from Models B to D than the results of Model A. Compared with Model A, Model E has the advantage of taking account of more parameters. However, it is weaker for not taking into account the number of victims. As the death risk is such a good indicator for other risks, we prefer Model A to Model E of the two simple models.

27. Is it at all ethically justified to apply the criterion “loss of income or of wealth”? We think it is because income are to some extent de facto used to increase the life expectancy or to prolong life. This is evident in poor countries where there is a lack of availability of medical treatments due to a lack of income. But even in some relatively rich countries the access to medical treatment depends on wealth or income. Finally, at the very high end, in the societies of the rich people in the rich countries, income or wealth are also invested in life-prolonging preventive screenings or treatments and better quality curative treatments. For these reasons, we prefer the ever more complex Models B to D to Model A. Evidently, Model D is the most comprehensive and complex, but also the most difficult to handle. The range from Model A to Model D is thus a range which offers ever more preciseness on one hand but requires ever more complex handling on the other. The right Model can only be chosen in the light of the capacity of the respective jurisdiction.

VI. Final remarks on the broader context

28. There are a variety of risks which have not been covered by this chart, e.g. risks linked to cognitive enhancement of humans or risks linked to the use of nano-technologies, an expression which covers a range of technical approaches and which suffers from intense debates on the right definition. The purpose of this table is not to show a complete list of risks, but to demonstrate how the various risks might be classified under the different Models. Regulators are invited to complete the list and, evidently, make their own assessment regarding the appropriate risk classes. We aim to demonstrate a methodology, not to prescribe or even recommend any results.

29. The risk classification is only one step in a multi-step process. We recommend proceeding in the order set-out below, but double-checking at the end of the process whether the right classification model has really been chosen. This has to be assessed in accordance with the results of the risk attribution and the corresponding sets of legal requirements. Some fine-tuning of the sets of legal requirements can conclude the process. We recommend thus the following order:

1. The selection and fine-tuning of classification models;

2. The attribution of the different risks to risk classes;

3. The determination of sets of legal requirements to be attributed in a proportionate way to the various risk classes.

4. The identification of exceptions to be made, e.g. exceptions at the level of the risk classification or exception with regard to the legal requirements. Evidently, if a need for very many exceptions is felt, the regulators should check whether something went wrong at the level of risk classification or establishment of legal requirements.

30. The method described in this blogpost and resumed in the previous paragraph can evidently be applied to other spheres as well. By no means is it limited to research and technology risks in general. The method can be transposed to other spheres as it can be downscaled to specific research undertakings or specific technologies.