Regulating Research and Technology Risks: Part I – Research Risks

This articles presents regulatory tools which can help to contain risks linked to research.

1. At university, I came to know a junior professor of physics. She was in charge of a synchrotron, a machine which accelerates particles to extremely high speed in order to cause collisions with obstacles or other particles. She admitted that neither her immediate colleagues in Bonn, at the time capital of the Federal Republic of Germany, nor the colleagues in Geneva which do basically the same thing at a much bigger scale could always predict what would happen during certain experiments. Surprising or even dangerous effects could not be ruled out. I asked her whether she had to comply with any regulatory requirements. She responded in the negative. In addition, she referred to a handful of nuclear research plants, partly also located close to city centres that were also widely exempted from regulation because they were managed by autonomous universities. Compared with these nuclear research plants, the little synchrotron was harmless. She told me that research was a kind of sacred zone that nobody would dare to impose regulation on.

2. This brings us to our topic. Research is done by private companies or quite autonomous public institutions, be they universities or specialised institutes. These institutions lack substantial external supervision.

3. This freedom from regulation is due to an asymmetry in expertise between these institutions and regulatory and legislative authorities. How can authorities assess risks to research when they do not understand it? But we need to raise two more fundamental questions:

– Are authorities informed on what is going on in the field of research?

– Are authorities empowered to investigate risks linked to research and take measures to contain the risks?

4. Our guess is that all three questions need to be answered in a way which is anything but comforting. At the same time, the risk associated with certain forms of research goes far beyond classic nuclear risks such as a melt-down of a nuclear plant. Areas with obvious risks difficult to control are geoengineering and synthetic biology. Another area of risk are ultra-resistant viruses, bacteria or other highly resistant forms of life escaping laboratories or being released by a terrorist or other criminal. A further concern is the risk associated with the possibility of self-replicating artificial intelligence. The potential damage of these developments is such that some of the best universities in the world have established specialised institutes to explore these risks, see e.g. the Centre for the Study of Existential Risk at the University of Cambridge. Publications on these topics often talk of these risks as if the technologies that pose them, and their progress, were an inevitability, whilst research institutions ignore the fact that the origin of many of these risks lies within research. Without research, most of the technologies would never appear. This paper considers these advances a result of current research, which falls into the domain of regulators to assess for potential risks, costs and benefits.

5. The regulation of research needs to drastically improve or to be created in the first place because:

– it is paradox that regulators regulate on so many minor risks whereas risks linked to research which, in extremis, might even endanger mankind are not appropriately or not at all regulated;

– it is possible to regulate research in the same way as other matters of society, as we will demonstrate below;

– the vast majority of research areas are of low risk potential and could be exempted so as to concentrate the administrative effort on high risk activities;

– even for those research activities which merit some scrutiny, the positive value of research is not negated by this assessment if it is conducted in a reasonable and proportionate way;

– reflection on risks of research and prudent regulation thereof might anticipate later discussions on risks of technologies, whilst risk steering can be implemented more effectively at research level than at the technology level;

– even where there is control of risks linked to technology, research risk control is needed because some risks arise specifically at the investigative stage while they are absent at the phase of application of a standardised technology;

– there is no reason why research should be privileged over technology in terms of risk control;

– the clock is ticking: if current research regulation is not improved upon, the scope of potential research activities is extremely large. As risk compounds over time, over the long term existential risks caused by current and future research poses a significant danger to mankind.

6. What could regulation on research look like? The answer to this question depends on the goals pursued by the regulator. We could imagine that the average regulator might have at least two goals:

A. Avoid the risk that research leads to the eradication of mankind;

B. Reduce other major risks for human beings to the extent that the expected positive effect of research is not disproportionately hampered.

Other goals might include:

C. Protection of animals;

D. Protection of nature.

Evidently, these goals need operationalisation and incentives for compliance need to be developed. A methodology for developing concrete regulations is outlined in the Handbook “How to regulate?” which is accessible in the right column of this website. Here we omit intermediate steps and present just the result of this exercise.

7. Which obligations could be set up for the companies undertaking research, research institutions and researchers? Companies undertaking research, research institutions and individual researchers could for instance be obliged:

(a) to assess risks prior to starting their research undertaking, with or without application of a relevant risk management standard;

(b) to reduce risks linked to the research undertaking to the extent possible if the risk reduction does not endanger the purpose of their research, with or without application of a relevant risk management standard;

(c) to refrain from research undertakings which trigger disproportionate / high risks;

(d) to inform the responsible authority of risks for health and life linked to a research undertaking;

(e) to request an authorisation from the responsible authority if the research undertaking triggers risks for health and life of a higher number of individuals;

(f) to request an authorisation from the responsible authority if the research undertaking triggers risks for the economic or ecological survival of the society in the jurisdiction in question or of societies in other jurisdictions;

(g) to register research projects in a database with two levels: public information and information only accessible to the authority in charge.

8. To ensure conformity with the major legal obligations, regulators might consider establishing procedural obligations for research institutions, such as undergoing certification to prove that the institutions’ internal processes ensure the fulfillment of these obligations. Risk management certification would be particularly helpful. Procedural obligations should be proportionate to the risks. To that end, it might be useful to establish risk classes and assign sets of (proportionate) procedural obligations to them.

9. To enable authorities to act, it is necessary that authorities are informed in advance or as soon as possible about potentially risky research projects. The information obligations contained in the previous paragraph might not be sufficient to ensure this result. The authorities might need to have access to work programs and documentation of universities and other research institutes on request. Therefore they should have comprehensive investigative powers. But in addition, they should have the means and, even more importantly, the scientific competence to assess the risks linked to research. To increase the effectiveness of authorities, authorities should be empowered to cooperate with their peers in the same and in other jurisdictions and this empowerment should include the right to transmit information on persons and confidential information relating to the research institutions or private companies undertaking research.

10. Once authorities have identified noteworthy risks, they need legal empowerments and work capacity to react to research projects which pose a particular risk. These measures might have a temporary or a definitive character. Two types of empowerment might be necessary. The range of measures to be taken by the authority should be as generic as legally possible in the respective jurisdiction because many different types of measures might be needed in the given case. However, the respective empowerments contained in the regulation should explicitly mention the most far reaching measures such as the confiscation of objects, including of computers and documents, the sealing of facilities and the destruction of harmful objects. In jurisdictions which require extremely precise and delimited empowerments, regulators might appreciate studying as reference or inspiration the Singapore Air Navigation (Amendment) Act 2014 which contains comprehensive empowerment in its Section 4.

In cases of extremely high risks, empowerments to supervise electronic and telecommunication might deemed to be appropriate, whilst a limitation of the individuals’ right to keep communication confidential might not be justified in cases of minor risks (principle of proportionality, applied at constitutional level in quite some jurisdictions).

11. When measures have been taken, the authority should have the legal power to communicate its decision to peers in the same and in other jurisdictions. This is necessary because nothing is

gained if risky research is just relocated to another jurisdiction. Publication of measures to peers might also stop a competition spiral downwards in terms of control intensity. Evidently administrations are expected to be research friendly. Accordingly, there might be a political wish to maintain a research and innovation friendly environment and consequently to ignore associated risks. Individual agents of administrations who wish to counter this pressure are in a better position if they can refer to measures taken in other jurisdictions. The exchange of information on considered or taken measures is thus very important for maintaining a climate which is open for justified risk limitation.

12. In case of research, particularly deterring empowerments could be considered: confiscation or transfer of patent and other intellectual property rights. The confiscation should also have effect at the level of private law so that the usual means for pursuing private rights in other jurisdictions can be used.

13. Evidently, a range of accompanying legal measures can be conceived. Regulators frequently consider penal sanctions against persons. They should also consider administrative sanctions against the legal entities organising the research and infringing legal obligations.

14. Legal enforcement and sanctions should be preceded by information campaigns, both in terms of fairness and because information alone might already change the behaviour of well intentioned researchers.

15. An often underestimated regulatory tool is the establishment of liability provisions. Once liability has been established, institutions concerned often start a kind of self-control process or look for insurances. Insurances will estimate the risks and may limit their insurance to cases where risk management is applied or other conditions are fulfilled. Evidently, this positive effect of the involvement of insurances can more systematically obtained by making liability insurance mandatory.

16. For high risk research projects, it is advisable that even the start of the project is subject to an authorisation. The authorisation mechanism ensures that the projects cannot start without screening by the administration. To prevent administrations from taking too long, it is possible to stipulate that the authorisation is deemed to be provided if the administration has not reacted within a certain time, e.g. two or three months.

17. The authorisation mechanism should provide some flexibility. E.g., regulators should ascertain that the authority is empowered to impose conditions when authorising the research project – a simple yes or no decision might not be appropriate in all situations. These conditions might need to cover the ways of execution or the scope of the research undertaking. The conditions might also be just of procedural nature, e.g. obliging the research institute to periodically report or to undergo a certain risk management certification based on the application of a generally recognised risk management standard. Authorities should also have the power to give authorisation for a defined period only, so that the authorisation needs to be renewed. Finally, authorities should be empowered to withdraw authorisations with retro-active effect in case of fraudulent or erroneous application and with effect for the future in case new information or newly discovered information leads to a negative assessment of the research project.

18. Regulators sometimes think that the authorisation mechanism makes classic empowerments as described under 9. to 15. superfluous. Experience shows that this is not the case. It should be checked which of the empowerments listed in 9. to 15. are still needed.

19. Whether authorisation requirements have been established or not, regulators should apply special care in order not to impose an unrealistic burden of proof on either of the two sides. Neither the research institutions will be able to provide full proof of safety, nor can authorities be expected to provide for such full proof of risk as justification for their measures. Hence it will be necessary to use unpleasantly vague formulas such as the following:

– Research projects shall be authorised if it is very unlikely that they trigger a major risk in the meaning of Article … . To demonstrate that their research project is very unlikely to trigger a major risk, research institutions shall submit scientific literature dealing directly or indirectly with risks of similar research projects.”

– Authorities may take measures against planned or ongoing research projects when there is either complete uncertainty regarding the risks triggered by the research project or if, based on first evidence or findings regarding similar research projects, it is not completely unlikely that the research project will trigger risks / major risks in the meaning of Article … .”

20. Further measures that could indirectly help the authorities include the following:

– Obligation of the authority to investigate risky research projects: such a legal obligation might help the authority to defend its interests when it comes to the annual budgeting exercise. Mandatory tasks can be easier defended against budget cuts.

– Requirements on independence and control of independence of the authority.

– Requirements on minimum resources of the authority (ideally with clear indication of minimum Full-time equivalences, not just vague clauses like „appropriate number of staff“ as the latter is difficult to enforce);

– Establishment of a research project register (with partly public and partly confidential information);

– Establishment of a research and technology risks observatory, ideally working for groups of jurisdictions (e.g. for several countries in one continent or for several states within the same Federal State. With “Federal State” we mean states like the U.S., India, Nigeria, Brazil, Germany where a good part of the state’s power is decentralised to sub-divisions.

– Establishment of a scientific advisory board.

– Obligation for (other) research institutions to make available their expertise to assess the risks of research projects.

– Creation of a central alert portal on which everybody may inform authorities about potentially problematic research.

– Whistle-blowing protection mechanisms protecting those who report in good faith to the authorities.

– Confidentiality provisions (assured confidentiality of information will increase readiness to cooperate with the authority).

– Cooperation agreements with other jurisdictions on information exchange, mutual advice, cooperation on enforcement etc.

21. It might also be suitable to use financial incentive to increase compliance. For research, public funds are the most important financial source. Hence it should be possible for authorities to establish a link between the fulfilment of legal obligations and of best practice codes going beyond legal obligations on one side and the public funding on the other, even if the authorities involved are not identical.

22. Tools of self-regulation might be helpful as well. We could imagine the following:

– Voluntary mutual control by analysis of research projects by an expert panel set-up by a roof organisation;

– Development of guidance and minimum standards by roof organisations or expert panels set-up by them;

– Control of the fulfilment of the minimum standards by the roof organisations or third bodies acting on their behalf.

23. Control of technologies is a slightly different topic than research risk control. But evidently, risks identified at the level of research should have a follow-up at the level of technology control. There are several reasons why a follow-up at the level of technology control is needed. First, banned research might be continued under the label technology development so as to (try to) circumvent the ban. Second, for the many cases where research is not banned but just subject to research risk control measures by authorities, it has to be analysed whether some of the control measures are also justified for the day-to-day application of the technology. We will look into this topic in one of the next blogposts on www.howtoregulate.org. Furthermore, we will check whether we can propose a model law for research and technology control.

Leave a Reply