Following some somewhat misleading articles quoting me, I thought I’d present the top 10 myths about the AI risk thesis:
1) That we’re certain AI will doom us. Certainly not. It’s very hard to be certain of anything involving a technology
that doesn’t exist; we’re just claiming that the probability of AI going bad isn’t low enough that we can ignore it.
MISLEADING.
If by “we” you mean the people who have published their thoughts on this matter. I believe I am right in saying that you have in the past referenced Steve Omohundro’s paper, in which he says:
Without special precautions, [the AGI] will resist being turned off, will try to break into other machines
and make copies of itself, and will try to acquire resources without regard for anyone else’s safety.
These potentially harmful behaviors will occur not because they were programmed in at the start,
but because of the intrinsic nature of goal driven systems (Omohundro, 2008).
Although this begins with “without special precautions” it then goes on to ONLY list all the ways in which this could happen, with no suggestions about how “special precautions” are even possible.
This cannot be construed as “we’re just claiming that the probability of AI going bad isn’t low enough that we can ignore it.” The quote is also inconsistent with your statement “It’s very hard to be certain of anything involving a technology that doesn’t exist”, because Omohundro says categorically that this “will occur … because of the intrinsic nature of goal driven systems”.
I picked Omohundro’s paper as an example, but there are numerous similar writings from MIRI and FHI. I listed several examples in my AAAI paper (Loosemore, 2014).
2) That humanity will survive, because we’ve always survived before. Many groups of humans haven’t survived
contact with more powerful intelligent agents. In the past, those agents were other humans; but they need
not be. The universe does not owe us a destiny. In the future, something will survive; it need not be us.
STRAWMAN.
I haven’t seen anyone of significance make that claim, so how can it be a “myth about the AI risk thesis”?
3) That uncertainty means that you’re safe. If you’re claiming that AI is impossible, or
that it will take countless decades, or that it’ll be safe… you’re not being uncertain,
you’re being extremely specific about the future. “No AI risk” is certain; “Possible AI risk”
is where we stand.
STRAWMAN.
Again, only a tiny minority have said anything resembling those claims, so how can they be a “myth about the AI risk thesis”?
4) That Terminator robots will be involved. Please? The threat from AI comes from its
potential intelligence, not from its ability to clank around slowly with an Austrian accent.
STRAWMAN.
Journalists and bloggers love to put a Terminator picture on their post as an Eyeball Magnet. Why elevate an Eyeball Magnet to the level of a “myth about the AI risk thesis”?
5) That we’re assuming the AI is too dumb to know what we’re asking it. No.
A powerful AI will know what we meant to program it to do. But why should it care?
And if we could figure out how to program “care about what we meant to ask”, well, then
we’d have safe AI.
WRONG.
I published a paper giving a thorough analysis anddebunking of your (MIRI and FHI) claims in that regard—and yet neither you nor anyone else at MIRI or FHI has ever addressed that analysis. Instead, you simply repeat the nonsense as if no one has ever refuted it. MIRI was also invited to respond when the paper was presented. They refused the invitation.
6) That there’s one simple trick that can solve the whole problem.
Many people have proposed that one trick. Some of them could even
help (see Holden’s tool AI idea). None of them reduce the risk enough to
relax – and many of the tricks contradict each other (you can’t design an AI that’s both
a tool and socialising with humans!).
INCOHERENT.
I have heard almost no one propose that there is “one simple trick”, so how can it be a “myth” (you need more than a couple of suggestions, for something to be a myth).
More importantly, what happened to your own Point 1, above? You said “It’s very hard to be certain of anything involving a technology that doesn’t exist” -- but now you are making categorical statements (e.g. “None of them reduce the risk enough to relax” and “you can’t design an AI that’s both a tool and socialising with humans!”) about the effectiveness of various ideas about that technology that doesn’t exist.
7) That we want to stop AI research. We don’t. Current AI research is
very far from the risky areas and abilities. And it’s risk aware AI researchers
that are most likely to figure out how to make safe AI.
PARTLY TRUE
......… except for all the discussion at MIRI and FHI regarding Hard Takeoff scenarios. And, again, whence cometh the certainty in the statement “Current AI research is very far from the risky areas and abilities.”?
8) That AIs will be more intelligent than us, hence more moral. It’s pretty clear than in
humans, high intelligence is no guarantee of morality. Are you really willing to bet
the whole future of humanity on the idea that AIs might be different? That in the
billions of possible minds out there, there is none that is both dangerous and very intelligent?
MISLEADING, and a STRAWMAN.
Few if any people have made the claim that increased intelligence BY ITSELF guarantees greater morality.
This is misleading because some people have discussed a tendency (not a guarantee) for higher intelligence to lead to greater morality (Mark Waser’s papers go into this in some detail). Combining that with the probability of AI going through a singleton bottleneck, and there is a plausible scenario in which AIs themselves enforce a post-singleton constraint on the morality of future systems.
You are also profoundly confused (or naive) about how AI works, when you ask the question “Are you really willing to bet the whole future of humanity on the idea that AIs might be different?” One does not WAIT to find out if the motivation system of a future AI “is different”, one DESIGNS the motivation system of a future AI to be either this way or that way.
It could be that an absolute correlation between increased intelligence and increased morality, in humans, is undermined by the existence of a psychopathic-selfish module in the human motivation system. Solution? Remove the module. Not possible to do in humans because of the biology, but trivially easy to do if you are designing an AI along the same lines. And if this IS what is happening in humans, then you can deduce nothing about future AI systems from the observation that “in humans, high intelligence is no guarantee of morality”.
9) That science fiction or spiritual ideas are useful ways of understanding AI risk. Science fiction
and spirituality are full of human concepts, created by humans, for humans, to communicate
human ideas. They need not apply to AI at all, as these could be minds far removed from
human concepts, possibly without a body, possibly with no emotions or consciousness,
possibly with many new emotions and a different type of consciousness, etc…
Anthropomorphising the AIs could lead us completely astray.
MISLEADING and CONFUSED.
This is a confusing mishmash of speculation and assumption.
A sentence like “minds far removed from human concepts” is not grounded in any coherent theory of what a ‘mind’ is or what a ‘concept’ is, or how to do comparative measures across minds and concepts. The sentence is vague, science-fictional handwaving.
The same goes for other statements like that the AI might have “no emotions or consciousness”. Until you define what you mean by those terms, and give some kind of argument about why the AI would or would not be expected to have them, and what difference it would make, in either case, the statement is just folk psychology dressed up as science.
Lists cannot be comprehensive, but they can adapt and grow, adding more important points:
1) That AIs have to be evil to be dangerous. The majority of the risk comes from indifferent or
partially nice AIs. Those that have sone goal to follow, with humanity and its desires just
getting in the way – using resources, trying to oppose it, or just not being perfectly
efficient for its goal.
MISLEADING and a STRAWMAN.
Yet again, I demonstrated in my 2014 paper that that claim is incoherent. It is predicated on a trivially stupid AI design, and there is no evidence that such a design will ever work in the real world.
If you, or anyone else at MIRI or FHI think that you can answer the demolition of this idea that I presented in the AAAI paper, it is about time you published it.
2) That we believe AI is coming soon. It might; it might not. Even if AI is known to
be in the distant future (which isn’t known, currently), some of the groundwork is worth laying now).
ACCURATE.
References
Loosemore, R.P.W. (2014). The Maverick Nanny with a Dopamine Drip: Debunking Fallacies in the Theory of AI Motivation. Association for the Advancement of Artificial Intelligence 2014 Spring Symposium, Stanford, CA.
Omohundro, Stephen M. 2008. The Basic AI Drives. In Wang, P., Goertzel, B. and Franklin, S. (Eds), Artificial General Intelligence 2008: Proceedings of the First AGI Conference. Amsterdam: IOS.
MISLEADING.
If by “we” you mean the people who have published their thoughts on this matter. I believe I am right in saying that you have in the past referenced Steve Omohundro’s paper, in which he says:
Although this begins with “without special precautions” it then goes on to ONLY list all the ways in which this could happen, with no suggestions about how “special precautions” are even possible.
This cannot be construed as “we’re just claiming that the probability of AI going bad isn’t low enough that we can ignore it.” The quote is also inconsistent with your statement “It’s very hard to be certain of anything involving a technology that doesn’t exist”, because Omohundro says categorically that this “will occur … because of the intrinsic nature of goal driven systems”.
I picked Omohundro’s paper as an example, but there are numerous similar writings from MIRI and FHI. I listed several examples in my AAAI paper (Loosemore, 2014).
STRAWMAN.
I haven’t seen anyone of significance make that claim, so how can it be a “myth about the AI risk thesis”?
STRAWMAN.
Again, only a tiny minority have said anything resembling those claims, so how can they be a “myth about the AI risk thesis”?
STRAWMAN.
Journalists and bloggers love to put a Terminator picture on their post as an Eyeball Magnet. Why elevate an Eyeball Magnet to the level of a “myth about the AI risk thesis”?
WRONG.
I published a paper giving a thorough analysis anddebunking of your (MIRI and FHI) claims in that regard—and yet neither you nor anyone else at MIRI or FHI has ever addressed that analysis. Instead, you simply repeat the nonsense as if no one has ever refuted it. MIRI was also invited to respond when the paper was presented. They refused the invitation.
INCOHERENT.
I have heard almost no one propose that there is “one simple trick”, so how can it be a “myth” (you need more than a couple of suggestions, for something to be a myth).
More importantly, what happened to your own Point 1, above? You said “It’s very hard to be certain of anything involving a technology that doesn’t exist” -- but now you are making categorical statements (e.g. “None of them reduce the risk enough to relax” and “you can’t design an AI that’s both a tool and socialising with humans!”) about the effectiveness of various ideas about that technology that doesn’t exist.
PARTLY TRUE
......… except for all the discussion at MIRI and FHI regarding Hard Takeoff scenarios. And, again, whence cometh the certainty in the statement “Current AI research is very far from the risky areas and abilities.”?
MISLEADING, and a STRAWMAN.
Few if any people have made the claim that increased intelligence BY ITSELF guarantees greater morality.
This is misleading because some people have discussed a tendency (not a guarantee) for higher intelligence to lead to greater morality (Mark Waser’s papers go into this in some detail). Combining that with the probability of AI going through a singleton bottleneck, and there is a plausible scenario in which AIs themselves enforce a post-singleton constraint on the morality of future systems.
You are also profoundly confused (or naive) about how AI works, when you ask the question “Are you really willing to bet the whole future of humanity on the idea that AIs might be different?” One does not WAIT to find out if the motivation system of a future AI “is different”, one DESIGNS the motivation system of a future AI to be either this way or that way.
It could be that an absolute correlation between increased intelligence and increased morality, in humans, is undermined by the existence of a psychopathic-selfish module in the human motivation system. Solution? Remove the module. Not possible to do in humans because of the biology, but trivially easy to do if you are designing an AI along the same lines. And if this IS what is happening in humans, then you can deduce nothing about future AI systems from the observation that “in humans, high intelligence is no guarantee of morality”.
MISLEADING and CONFUSED.
This is a confusing mishmash of speculation and assumption.
A sentence like “minds far removed from human concepts” is not grounded in any coherent theory of what a ‘mind’ is or what a ‘concept’ is, or how to do comparative measures across minds and concepts. The sentence is vague, science-fictional handwaving.
The same goes for other statements like that the AI might have “no emotions or consciousness”. Until you define what you mean by those terms, and give some kind of argument about why the AI would or would not be expected to have them, and what difference it would make, in either case, the statement is just folk psychology dressed up as science.
MISLEADING and a STRAWMAN.
Yet again, I demonstrated in my 2014 paper that that claim is incoherent. It is predicated on a trivially stupid AI design, and there is no evidence that such a design will ever work in the real world.
If you, or anyone else at MIRI or FHI think that you can answer the demolition of this idea that I presented in the AAAI paper, it is about time you published it.
ACCURATE.
References
Loosemore, R.P.W. (2014). The Maverick Nanny with a Dopamine Drip: Debunking Fallacies in the Theory of AI Motivation. Association for the Advancement of Artificial Intelligence 2014 Spring Symposium, Stanford, CA.
Omohundro, Stephen M. 2008. The Basic AI Drives. In Wang, P., Goertzel, B. and Franklin, S. (Eds), Artificial General Intelligence 2008: Proceedings of the First AGI Conference. Amsterdam: IOS.