I am not making an argument about regulation in general here. I’m talking within the context of TRIZ-Ingenieur’s comment and the first point is that regulation will not save him from the dangers that he’s afraid of.
I don’t see how it works even in this specific context. TRIZ-Ingenieur hopes that regulation of AI research could reduce dangerous AI research. All regulation of dangerous things simply redistributes power between fallible humans; that’s no truer of AI research regulation than it is of any other regulation.
(It may be that there are special reasons why AI research is particularly unsuited for attempts to make it safer by regulation, but you didn’t mention any or even allude to any.)
The implication I’m reading in TRIZ-Ingenieur’s words is that humans are weak, fallible, corruptible—but a regulatory body is not. To quote him,
The regulatory body takes power away from the fallible human
This is a common fallacy where some body (organization, committee, council, etc.) is considered to be immune to human weaknesses as if it were composed of selfless enlightened philosopher-kings.
Essentially, the argument here is that mere humans can’t be trusted with AI development. Without opining on the truth of the subject claim, my point is that if they can’t, having a regulatory body won’t help.
My idea of a regulatory body is not that of a powerful institution that it deeply interacts with all ongoing projects because of the known fallible members who could misuse their power.
My idea of a regulatory body could be more that of a TÜV interconnected with institutions who do AI safety research and develop safety standards, test methods and test data. Going back to the TÜVs foundation task: pressure vessel certification. Any qualified test institution in the world can check if it is safe to use a given pressure vessel based on established design tests, safety measures checks, material testing methods and real pressure check tests. The amount of safety measures, tests and certification effort depends on the danger potential (pressure, volume, temperature, medium). Standards define based on danger potential and application which of the following safety measures must be used: safety valve; rupture disk; pressure limiter, temperature limiter, liquid indicator, overfill protection; vacuum breakers; reaction blocker; water sprinkling devices.
Nick Bostrum named following AI safety measures: boxing methods, incentive methods, stunting and tripwires.
Pressure vessels and AI have following common elements (AI related argument plausible, but no experience exists):
Human casualties are result of a bursting vessel or AI turning evil.
Good design, tests and safety measures reduce risk of failing.
Humans want to use both.
Companies, institutions and legislation had 110 years of development and improvement of standards for pressure vessels. With AI we are still scratching on the surface. AI and pressure vessels have following differences:
Early designs of pressure vessels were prone to burst—AI is stil far away from high risk level.
Many bursting vessel events successively stimulated improvement of standards—With AI the first singularity will be the only one.
Safety measures of pressure vessels are easily comprehensible—Easy AI safety measures reduce its functionality to a high degree, complex safety measures allow full functionality but are complex to implement, complex to test and to standardize.
The risk of a bursting pressure vessel is obvious—the risk of an evil Singularity is opaque and diffuse.
Safety measure research for pressure vessels is straight forward following physical laws—safety research for AI is a multifaceted cloud of concepts.
A bursting pressure vessel may kill a few dozen people—an evil Singularity might eradicate humankind.
Given the existential risk of AI I think most AI research institutions could agree on a code of conduct that would include e.g.
AIs will be classified in danger classes. The rating depends on computational power, taught knowledge areas, degree of self-optimization capacity. An AI with programming and hacking abilities will be classified as high risk application even if it is running on moderate hardware because of its intrinsic capabilities to escape into the cloud.
The amount of necessary safety measures depends on this risk rating:
Low risk applications have to be firewalled against acquisition of computing power in other computers.
Medium risk applications must additionally have internal safety measures e.g. stunting or tripwires.
High risk applications in addition must be monitored internally and externally by independently developed tool AIs.
Design and safeguard measures of medium and high risk applications will be independently checked and pentested by independent safety institutions.
In a first step safety AI research institutes develop monitoring AIs, tool AIs, pentesting datasets and finally guidelines like the one above.
In a second step public financed AI projects have to follow these guidelines. This applies to university projects in particular.
Public pressure and stockholders could push companies to apply these guidelines. Maybe an ISO certificate can indicate to the public: “All AI projects of this company follow the ISO Standard for AI risk assessment and safeguard measures”
The public opinion and companies hopefully will push governments to enforce these guidelines as well within their intelligence agencies. A treaty in the mind of the Non-Proliferation Treaty could be signed. All signing states ensure to obey the ISO Standard on AI within their institutions.
I accept that there are many IFs and obstacles on that path. But it is at least an IDEA how civil society can push AI developers to implement safeguards into their designs.
How many researchers join the AI field will only marginally change the acceleration of computing power. If only a few people work on AI they have enough to do to grab all the low-hanging fruit. If many join AI research more meta research and safety research will be possible. If only a fraction of this depicted path will turn into reality it will give jobs to some hundred researchers.
I agree that if TRIZ-Ingenieur thinks regulatory bodies are strong, infallible, and incorruptible, then he is wrong. I don’t see any particular reason to think he thinks that, though. It may in fact suffice for regulatory bodies’ weaknesses, errors and corruptions to be different from those of the individual humans being regulated, which they often are.
(I do not get the impression that T-I thinks “mere humans can’t be trusted with AI development” in any useful sense[1].)
[1] Example of a not-so-useful sense: it is probably true that mere humans can’t with 100% confidence of safety be trusted with AI development, or with anything else, and indeed the same will be true of regulatory bodies. But this doesn’t yield a useful argument against AI development for anyone who cares about averages and probabilities rather than only about the very worst case.
I am not making an argument about regulation in general here. I’m talking within the context of TRIZ-Ingenieur’s comment and the first point is that regulation will not save him from the dangers that he’s afraid of.
I don’t see how it works even in this specific context. TRIZ-Ingenieur hopes that regulation of AI research could reduce dangerous AI research. All regulation of dangerous things simply redistributes power between fallible humans; that’s no truer of AI research regulation than it is of any other regulation.
(It may be that there are special reasons why AI research is particularly unsuited for attempts to make it safer by regulation, but you didn’t mention any or even allude to any.)
The implication I’m reading in TRIZ-Ingenieur’s words is that humans are weak, fallible, corruptible—but a regulatory body is not. To quote him,
This is a common fallacy where some body (organization, committee, council, etc.) is considered to be immune to human weaknesses as if it were composed of selfless enlightened philosopher-kings.
Essentially, the argument here is that mere humans can’t be trusted with AI development. Without opining on the truth of the subject claim, my point is that if they can’t, having a regulatory body won’t help.
My idea of a regulatory body is not that of a powerful institution that it deeply interacts with all ongoing projects because of the known fallible members who could misuse their power.
My idea of a regulatory body could be more that of a TÜV interconnected with institutions who do AI safety research and develop safety standards, test methods and test data. Going back to the TÜVs foundation task: pressure vessel certification. Any qualified test institution in the world can check if it is safe to use a given pressure vessel based on established design tests, safety measures checks, material testing methods and real pressure check tests. The amount of safety measures, tests and certification effort depends on the danger potential (pressure, volume, temperature, medium). Standards define based on danger potential and application which of the following safety measures must be used: safety valve; rupture disk; pressure limiter, temperature limiter, liquid indicator, overfill protection; vacuum breakers; reaction blocker; water sprinkling devices.
Nick Bostrum named following AI safety measures: boxing methods, incentive methods, stunting and tripwires. Pressure vessels and AI have following common elements (AI related argument plausible, but no experience exists):
Human casualties are result of a bursting vessel or AI turning evil.
Good design, tests and safety measures reduce risk of failing.
Humans want to use both.
Companies, institutions and legislation had 110 years of development and improvement of standards for pressure vessels. With AI we are still scratching on the surface. AI and pressure vessels have following differences:
Early designs of pressure vessels were prone to burst—AI is stil far away from high risk level.
Many bursting vessel events successively stimulated improvement of standards—With AI the first singularity will be the only one.
Safety measures of pressure vessels are easily comprehensible—Easy AI safety measures reduce its functionality to a high degree, complex safety measures allow full functionality but are complex to implement, complex to test and to standardize.
The risk of a bursting pressure vessel is obvious—the risk of an evil Singularity is opaque and diffuse.
Safety measure research for pressure vessels is straight forward following physical laws—safety research for AI is a multifaceted cloud of concepts.
A bursting pressure vessel may kill a few dozen people—an evil Singularity might eradicate humankind.
Given the existential risk of AI I think most AI research institutions could agree on a code of conduct that would include e.g.
AIs will be classified in danger classes. The rating depends on computational power, taught knowledge areas, degree of self-optimization capacity. An AI with programming and hacking abilities will be classified as high risk application even if it is running on moderate hardware because of its intrinsic capabilities to escape into the cloud.
The amount of necessary safety measures depends on this risk rating:
Low risk applications have to be firewalled against acquisition of computing power in other computers.
Medium risk applications must additionally have internal safety measures e.g. stunting or tripwires.
High risk applications in addition must be monitored internally and externally by independently developed tool AIs.
Design and safeguard measures of medium and high risk applications will be independently checked and pentested by independent safety institutions.
In a first step safety AI research institutes develop monitoring AIs, tool AIs, pentesting datasets and finally guidelines like the one above.
In a second step public financed AI projects have to follow these guidelines. This applies to university projects in particular.
Public pressure and stockholders could push companies to apply these guidelines. Maybe an ISO certificate can indicate to the public: “All AI projects of this company follow the ISO Standard for AI risk assessment and safeguard measures”
The public opinion and companies hopefully will push governments to enforce these guidelines as well within their intelligence agencies. A treaty in the mind of the Non-Proliferation Treaty could be signed. All signing states ensure to obey the ISO Standard on AI within their institutions.
I accept that there are many IFs and obstacles on that path. But it is at least an IDEA how civil society can push AI developers to implement safeguards into their designs.
How many researchers join the AI field will only marginally change the acceleration of computing power. If only a few people work on AI they have enough to do to grab all the low-hanging fruit. If many join AI research more meta research and safety research will be possible. If only a fraction of this depicted path will turn into reality it will give jobs to some hundred researchers.
I agree that if TRIZ-Ingenieur thinks regulatory bodies are strong, infallible, and incorruptible, then he is wrong. I don’t see any particular reason to think he thinks that, though. It may in fact suffice for regulatory bodies’ weaknesses, errors and corruptions to be different from those of the individual humans being regulated, which they often are.
(I do not get the impression that T-I thinks “mere humans can’t be trusted with AI development” in any useful sense[1].)
[1] Example of a not-so-useful sense: it is probably true that mere humans can’t with 100% confidence of safety be trusted with AI development, or with anything else, and indeed the same will be true of regulatory bodies. But this doesn’t yield a useful argument against AI development for anyone who cares about averages and probabilities rather than only about the very worst case.