I am someone who is at present unsure how to think about AI risk. As a complete layperson with a strong interest in science, technology, futurism and so on, there are—seemingly—some very smart people in the field who appear to be saying that the risk is basically zero (eg: Andrew Ng, Yann Le Cunn). Then there are others who are very worried indeed—as represented by this post I am responding to.
This is confusing.
To get people at my level to support a shut down of the type described above, there needs to be some kind of explanation as to why there is such a difference of opinion by experts, because any argument that you make to me to accept AI as a risk that requires such a shut down has been both rejected and accepted by others who know more than me about AI.
Note that this may not be rationality as it is understood on this forum—after all, why can’t I just weigh the arguments without looking at who supports them? I understand that. But if I am sceptical about my own reasoning capabilities in this area—given that I am a lay person—then I have to suspect that any argument (either for or against AI risk) that has not convinced people with superior reasoning capabilities than I may contain flaws.
That is, unless I understand why there might be such disagreement.
And I understand that this might get into recursion—people disagree about the reasons for disagreement and …
However, at the very least it gives me another lens to look through and also someone with a lot of knowledge in AI might not have a lot of knowledge in why arguments fail.
Yes, it’s a difficult problem for a layman to know how alarmed to be. I’m in the AI field, and I’ve thought that superhuman AI was a threat since about 2003. I’d be glad to engage you in an offline object-level discussion about it, comprehensible to a layman, if you think that would help. I have some experience in this, having engaged in many such discussions. It’s not complicated or technical, if you explain it right.
I don’t have a general theory for why people disagree with me, but here are several counter arguments I have encountered. I phrase them as though they were being suggested to me, so “you” is actually me.
— Robots taking over sounds nuts, so you must be crazy.
— This is an idea from a science fiction movie. You’re not a serious person.
— People often predict the end of the world, and they’ve always been wrong before. And often been psychologically troubled. Are you seeing a therapist?
— Why don’t any of the top people in your field agree? Surely if this were a serious problem, they’d be all over it. (don’t hear this one much any more.)
— AIs won’t be dangerous, because nobody would be so foolish as to design them that way. Or to build AIs capable of long term planning, or to direct AIs toward foolish or harmful goals. Or various other sentences containing the phrase “nobody would be so foolish as to”. — AIs will have to obey the law, so we don’t have to worry about them killing people or taking over, because those things are illegal. (Yes, I’ve actually heard this one.)
— Various principles of computer science show that it is impossible to build a machine that makes correct choices in all circumstances. (This is where the “no free lunch“ theorem comes in. Of course, we’re not proposing a machine that makes correct choices in all circumstances, just one that makes mostly correct choices in the circumstances it encounters.)
— There will be lots of AIs, and the good ones will outnumber the bad ones and hence win.
— It’s impossible to build a machine with greater-than-human intelligence, because of <philosophical principle here>.
— Greater wisdom leads to greater morality, so a superhuman AI is guaranteed beneficent.
— If an AI became dangerous, I would just unplug it. Yes, I’d be able to spot it, and no, the AI wouldn’t be able to talk me out of it, or otherwise stop me.
— Machines can never become conscious. Which implies safety, somehow.
— Present-day AIs are obviously not able to take over the world. They’re not even scary. You’re foolishly over-reacting.
— The real problem of AI is <something else, usually something already happening>. You’re distracting people with your farfetched speculation.
— My whole life, people have been decrying technological advances and saying they were bad, and they’ve always been wrong. You must be one of those Luddites we keep hearing about.
— If it becomes a problem, people will take care of it.
— My paycheck depends on my not agreeing with you. (I’ve been working on this one— convincing my friends in the AI business to retreat from frontier development. Results are mixed.)
— Superhuman machines offer vast payoff! We must press ahead regardless.
— If humans are defeated, that’s good actually, because evolution is good.
Many of these are good arguments, but unfortunately they’re all wrong.
I tend to concentrate on extinction, as the most massive and terrifying of risks. I think that smaller problems can be dealt with by the usual methods, like our society has dealt with lots of things. Which is not to say that they aren’t real problems, that do real harm, and require real solutions. My disagreement is with “You’re distracting people with your farfetched speculation.” I don’t think raising questions of existential risk makes it harder to deal with more quotidian problems. And even if it did, that’s not an argument against the reality of extinction risk.
To me the core reason for wide disagreement seems simple enough—at this stage the essential nature of AI existential risk arguments is not scientific but philosophical. The terms are informal and there are no grounded models of underlying dynamics (in contrast with e.g. climate change). Large persistent philosophical disagreements are very much the widespread norm, and thus unsurprising in this particular instance as well, even among experts in currently existing AIs, as it’s far from clear how their insights would extrapolate to hypothetical future systems.
there needs to be some kind of explanation as to why there is such a difference of opinion by experts
Isn’t this kind of thing the default? Like, for ~every invention that changed the world I’d expect to be able to find experts saying in advance that it won’t work or if it does it won’t change things much. And for lots of things that didn’t work or didn’t change the world, I’d expect to be able to find experts saying it would. I basically just think that “smart person believes silly thing for silly reasons” is pretty common.
True. Unless there were very good arguments/very good evidence for one side or the other. My expectation is that for any random hypothesis there will be lots of disagreement about it among experts. For a random hypothesis with lots of good arguments/good evidence, I would expect much, much less disagreement among experts in the field.
If we look at climate change, for example, the vast majority of experts agreed about it quite early on—within 15 years of the Charney report.
If all I am left with, however, is ‘smart person believes silly thing for silly reasons’ then it is not reasonable for me as a lay person to determine which is the silly thing. Is ‘AI poses no (or extremely low) x-risk’ the silly thing, or is ‘AI poses unacceptable x-risk’ the silly thing?
If AI does indeed pose unacceptable x-risk and there are good arguments/good evidence for this, then there also has to be a good reason or set of reasons why many experts are not convinced. (Yann claims, for example, that the AI experts arguing for AI x-risk are a very small minority and Eliezer Yudkowsky seems to agree with this).
If we look at climate change, for example, the vast majority of experts agreed about it quite early on—within 15 years of the Charney report.
So I don’t know much about timelines of global warming or global warming science, but I note that that report came out in 1979, more than 100 years after the industrial revolution. So it’s not clear to me that fifteen years after that counts as “quite early on”, or that AI science is currently at a comparable point in the timeline. (If points in these timelines can even be compared.)
If all I am left with, however, is ‘smart person believes silly thing for silly reasons’ then it is not reasonable for me as a lay person to determine which is the silly thing.
FWIW I think even relatively-lay people can often detect silly arguments, even from people who know a lot more than them. Some examples where I think I’ve done that:
I remember seeing someone (possibly even Yann LeCun?) saying something along the lines of, AGI is impossible because of no free lunch theorems.
Someone saying that HPMOR’s “you violated conservation of energy!” bit is dumb because something something quantum stuff that I didn’t understand; and also because if turning into a cat violated conservation of energy, then so did levitating someone a few paragraphs earlier. I am confident this person (who went by the handle su3su2u1)
knows a lot more about physics than me. I am also confident this second part was them being silly.
So I’d suggest that you might be underestimating yourself.
But if you’re right that you can’t reasonably figure this out… I’m not sure there are any ways to get around that? Eliezer can say “Yann believes this because of optimism bias” and Yann can say “Eliezer believes this because of availability heuristic” or whatever, and maybe one or both of them is right (tbc I have not observed either of them saying these things). But these are both Bulverism.
It may be that Eliezer and Yann can find a double crux, something where they agree: “Eliezer believes X, and if Eliezer believed not-X then Eliezer would think AGI does not pose a serious risk. Yann believes not-X, and if Yann believed X then Yann would think AGI does pose a serious risk.” But finding such Xs is hard, I don’t expect there to be a simple one, and even if there was it just punts the question: “why do these two smart people disagree on X?” It’s possible X is in a domain that you consider yourself better able to have an opinion on, but it’s also possible it’s in one you consider yourself less able to have an opinion on.
If AI does indeed pose unacceptable x-risk and there are good arguments/good evidence for this, then there also has to be a good reason or set of reasons why many experts are not convinced.
I basically just don’t think there does have to be this.
(Yann claims, for example, that the AI experts arguing for AI x-risk are a very small minority and Eliezer Yudkowsky seems to agree with this)
Fwiw my sense is that this is false, and that Yann might believe it but I don’t expect Eliezer to. But I don’t remember what I’ve seen that makes me think this. (To some extent it might depend on who you count as an expert and what you count as arguing for x-risk.)
Re timelines for climate change, in the 1970s, serious people in the field of climate studies started suggesting that there was a serious problem looming. A very short time later, the entire field was convinced by the evidence and argument for that serious risk—to the point that the IPCC was established in 1988 by the UN.
When did some serious AI researchers start to suggest that there was a serious problem looming? I think in the 2000s. There is no IPAIX-risk.
And, yes: I can detect silly arguments in a reasonable number of cases. But I have not been able to do so in this case as yet (in the aggregate). It seems that there are possibly good arguments on both sides.
It is indeed tricky—I also mentioned that it could get into a regress-like situation. But I think that if people like me are to be convinced it might be worth the attempt. As you say, there may be a more accessible to me domain in there somewhere.
Re the numbers, Eliezer seems to claim that the majority of AI researchers believe in X-risk, but few are speaking out for a variety of reasons. This boils down to me trusting Eliezer’s word about the majority belief, because that majority is not speaking out. He may be motivated to lie in this case—note that I am not saying that he is, but ‘lying for Jesus’ (for example) is a relatively common thing. It is also possible that he is not lying but is wrong—he may have talked to a sample that was biased in some way.
Re timelines for climate change, in the 1970s, serious people in the field of climate studies started suggesting that there was a serious problem looming. A very short time later, the entire field was convinced by the evidence and argument for that serious risk—to the point that the IPCC was established in 1988 by the UN.
When did some serious AI researchers start to suggest that there was a serious problem looming? I think in the 2000s. There is no IPAIX-risk.
Nod. But then, I assume by the 1970s there was already observable evidence of warming? Whereas the observable evidence of AI X-risk in the 2000s seems slim. Like I expect I could tell a story for global warming along the lines of “some people produced a graph with a trend line, and some people came up with theories to explain it”, and for AI X-risk I don’t think we have graphs or trend lines of the same quality.
This isn’t particularly a crux for me btw. But like, there are similarities and differences between these two things, and pointing out the similarities doesn’t really make me expect that looking at one will tell us much about the other.
I think that if people like me are to be convinced it might be worth the attempt. As you say, there may be a more accessible to me domain in there somewhere.
Not opposed to trying, but like...
So I think it’s basically just good to try to explain things more clearly and to try to get to the roots of disagreements. There are lots of ways this can look like. We can imagine a conversation between Eliezer and Yann, or people who respectively agree with them. We can imagine someone currently unconvinced having individual conversations with each side. We can imagine discussions playing out through essays written over the course of months. We can imagine FAQs written by each side which give their answers to the common objections raised by the other. I like all these things.
And maybe in the process of doing these things we eventually find a “they disagree because …” that helps it click for you or for others.
What I’m skeptical about is trying to explain the disagreement rather than discover it. That is, I think “asking Eliezer to explain what’s wrong with Yann’s arguments” works better than “asking Eliezer to explain why Yann disagrees with him”. I think answers I expect to the second question basically just consist of “answers I expect to the first question” plus “Bulverism”.
(Um, having written all that I realize that you might just have been thinking of the same things I like, and describing them in a way that I wouldn’t.)
Unfortunately I do not know the reasoning behind why the people you mentioned might not see AI as a threat, but if I had to guess – people not worried are primarily thinking about short term AI safety risks like disinformation from deepfakes, and people worried are thinking about super-intelligent AGI and instrumental convergence, which necessitates solving the alignment problem.
I am someone who is at present unsure how to think about AI risk. As a complete layperson with a strong interest in science, technology, futurism and so on, there are—seemingly—some very smart people in the field who appear to be saying that the risk is basically zero (eg: Andrew Ng, Yann Le Cunn). Then there are others who are very worried indeed—as represented by this post I am responding to.
This is confusing.
To get people at my level to support a shut down of the type described above, there needs to be some kind of explanation as to why there is such a difference of opinion by experts, because any argument that you make to me to accept AI as a risk that requires such a shut down has been both rejected and accepted by others who know more than me about AI.
Note that this may not be rationality as it is understood on this forum—after all, why can’t I just weigh the arguments without looking at who supports them? I understand that. But if I am sceptical about my own reasoning capabilities in this area—given that I am a lay person—then I have to suspect that any argument (either for or against AI risk) that has not convinced people with superior reasoning capabilities than I may contain flaws.
That is, unless I understand why there might be such disagreement.
And I understand that this might get into recursion—people disagree about the reasons for disagreement and …
However, at the very least it gives me another lens to look through and also someone with a lot of knowledge in AI might not have a lot of knowledge in why arguments fail.
Yes, it’s a difficult problem for a layman to know how alarmed to be. I’m in the AI field, and I’ve thought that superhuman AI was a threat since about 2003. I’d be glad to engage you in an offline object-level discussion about it, comprehensible to a layman, if you think that would help. I have some experience in this, having engaged in many such discussions. It’s not complicated or technical, if you explain it right.
I don’t have a general theory for why people disagree with me, but here are several counter arguments I have encountered. I phrase them as though they were being suggested to me, so “you” is actually me.
— Robots taking over sounds nuts, so you must be crazy.
— This is an idea from a science fiction movie. You’re not a serious person.
— People often predict the end of the world, and they’ve always been wrong before. And often been psychologically troubled. Are you seeing a therapist?
— Why don’t any of the top people in your field agree? Surely if this were a serious problem, they’d be all over it. (don’t hear this one much any more.)
— AIs won’t be dangerous, because nobody would be so foolish as to design them that way. Or to build AIs capable of long term planning, or to direct AIs toward foolish or harmful goals. Or various other sentences containing the phrase “nobody would be so foolish as to”.
— AIs will have to obey the law, so we don’t have to worry about them killing people or taking over, because those things are illegal. (Yes, I’ve actually heard this one.)
— Various principles of computer science show that it is impossible to build a machine that makes correct choices in all circumstances. (This is where the “no free lunch“ theorem comes in. Of course, we’re not proposing a machine that makes correct choices in all circumstances, just one that makes mostly correct choices in the circumstances it encounters.)
— There will be lots of AIs, and the good ones will outnumber the bad ones and hence win.
— It’s impossible to build a machine with greater-than-human intelligence, because of <philosophical principle here>.
— Greater wisdom leads to greater morality, so a superhuman AI is guaranteed beneficent.
— If an AI became dangerous, I would just unplug it. Yes, I’d be able to spot it, and no, the AI wouldn’t be able to talk me out of it, or otherwise stop me.
— Machines can never become conscious. Which implies safety, somehow.
— Present-day AIs are obviously not able to take over the world. They’re not even scary. You’re foolishly over-reacting.
— The real problem of AI is <something else, usually something already happening>. You’re distracting people with your farfetched speculation.
— My whole life, people have been decrying technological advances and saying they were bad, and they’ve always been wrong. You must be one of those Luddites we keep hearing about.
— If it becomes a problem, people will take care of it.
— My paycheck depends on my not agreeing with you. (I’ve been working on this one— convincing my friends in the AI business to retreat from frontier development. Results are mixed.)
— Superhuman machines offer vast payoff! We must press ahead regardless.
— If humans are defeated, that’s good actually, because evolution is good.
Many of these are good arguments, but unfortunately they’re all wrong.
I am happy to have a conversation with you. On this point:
‘— The real problem of AI is <something else, usually something already happening>. You’re distracting people with your farfetched speculation.’
I believe that AI indeed poses huge problems, so maybe this is where I sit.
I tend to concentrate on extinction, as the most massive and terrifying of risks. I think that smaller problems can be dealt with by the usual methods, like our society has dealt with lots of things. Which is not to say that they aren’t real problems, that do real harm, and require real solutions. My disagreement is with “You’re distracting people with your farfetched speculation.” I don’t think raising questions of existential risk makes it harder to deal with more quotidian problems. And even if it did, that’s not an argument against the reality of extinction risk.
To me the core reason for wide disagreement seems simple enough—at this stage the essential nature of AI existential risk arguments is not scientific but philosophical. The terms are informal and there are no grounded models of underlying dynamics (in contrast with e.g. climate change). Large persistent philosophical disagreements are very much the widespread norm, and thus unsurprising in this particular instance as well, even among experts in currently existing AIs, as it’s far from clear how their insights would extrapolate to hypothetical future systems.
Isn’t this kind of thing the default? Like, for ~every invention that changed the world I’d expect to be able to find experts saying in advance that it won’t work or if it does it won’t change things much. And for lots of things that didn’t work or didn’t change the world, I’d expect to be able to find experts saying it would. I basically just think that “smart person believes silly thing for silly reasons” is pretty common.
True. Unless there were very good arguments/very good evidence for one side or the other. My expectation is that for any random hypothesis there will be lots of disagreement about it among experts. For a random hypothesis with lots of good arguments/good evidence, I would expect much, much less disagreement among experts in the field.
If we look at climate change, for example, the vast majority of experts agreed about it quite early on—within 15 years of the Charney report.
If all I am left with, however, is ‘smart person believes silly thing for silly reasons’ then it is not reasonable for me as a lay person to determine which is the silly thing. Is ‘AI poses no (or extremely low) x-risk’ the silly thing, or is ‘AI poses unacceptable x-risk’ the silly thing?
If AI does indeed pose unacceptable x-risk and there are good arguments/good evidence for this, then there also has to be a good reason or set of reasons why many experts are not convinced. (Yann claims, for example, that the AI experts arguing for AI x-risk are a very small minority and Eliezer Yudkowsky seems to agree with this).
So I don’t know much about timelines of global warming or global warming science, but I note that that report came out in 1979, more than 100 years after the industrial revolution. So it’s not clear to me that fifteen years after that counts as “quite early on”, or that AI science is currently at a comparable point in the timeline. (If points in these timelines can even be compared.)
FWIW I think even relatively-lay people can often detect silly arguments, even from people who know a lot more than them. Some examples where I think I’ve done that:
I remember seeing someone (possibly even Yann LeCun?) saying something along the lines of, AGI is impossible because of no free lunch theorems.
Someone saying that HPMOR’s “you violated conservation of energy!” bit is dumb because something something quantum stuff that I didn’t understand; and also because if turning into a cat violated conservation of energy, then so did levitating someone a few paragraphs earlier. I am confident this person (who went by the handle su3su2u1) knows a lot more about physics than me. I am also confident this second part was them being silly.
This comment.
So I’d suggest that you might be underestimating yourself.
But if you’re right that you can’t reasonably figure this out… I’m not sure there are any ways to get around that? Eliezer can say “Yann believes this because of optimism bias” and Yann can say “Eliezer believes this because of availability heuristic” or whatever, and maybe one or both of them is right (tbc I have not observed either of them saying these things). But these are both Bulverism.
It may be that Eliezer and Yann can find a double crux, something where they agree: “Eliezer believes X, and if Eliezer believed not-X then Eliezer would think AGI does not pose a serious risk. Yann believes not-X, and if Yann believed X then Yann would think AGI does pose a serious risk.” But finding such Xs is hard, I don’t expect there to be a simple one, and even if there was it just punts the question: “why do these two smart people disagree on X?” It’s possible X is in a domain that you consider yourself better able to have an opinion on, but it’s also possible it’s in one you consider yourself less able to have an opinion on.
I basically just don’t think there does have to be this.
Fwiw my sense is that this is false, and that Yann might believe it but I don’t expect Eliezer to. But I don’t remember what I’ve seen that makes me think this. (To some extent it might depend on who you count as an expert and what you count as arguing for x-risk.)
Re timelines for climate change, in the 1970s, serious people in the field of climate studies started suggesting that there was a serious problem looming. A very short time later, the entire field was convinced by the evidence and argument for that serious risk—to the point that the IPCC was established in 1988 by the UN.
When did some serious AI researchers start to suggest that there was a serious problem looming? I think in the 2000s. There is no IPAIX-risk.
And, yes: I can detect silly arguments in a reasonable number of cases. But I have not been able to do so in this case as yet (in the aggregate). It seems that there are possibly good arguments on both sides.
It is indeed tricky—I also mentioned that it could get into a regress-like situation. But I think that if people like me are to be convinced it might be worth the attempt. As you say, there may be a more accessible to me domain in there somewhere.
Re the numbers, Eliezer seems to claim that the majority of AI researchers believe in X-risk, but few are speaking out for a variety of reasons. This boils down to me trusting Eliezer’s word about the majority belief, because that majority is not speaking out. He may be motivated to lie in this case—note that I am not saying that he is, but ‘lying for Jesus’ (for example) is a relatively common thing. It is also possible that he is not lying but is wrong—he may have talked to a sample that was biased in some way.
Nod. But then, I assume by the 1970s there was already observable evidence of warming? Whereas the observable evidence of AI X-risk in the 2000s seems slim. Like I expect I could tell a story for global warming along the lines of “some people produced a graph with a trend line, and some people came up with theories to explain it”, and for AI X-risk I don’t think we have graphs or trend lines of the same quality.
This isn’t particularly a crux for me btw. But like, there are similarities and differences between these two things, and pointing out the similarities doesn’t really make me expect that looking at one will tell us much about the other.
Not opposed to trying, but like...
So I think it’s basically just good to try to explain things more clearly and to try to get to the roots of disagreements. There are lots of ways this can look like. We can imagine a conversation between Eliezer and Yann, or people who respectively agree with them. We can imagine someone currently unconvinced having individual conversations with each side. We can imagine discussions playing out through essays written over the course of months. We can imagine FAQs written by each side which give their answers to the common objections raised by the other. I like all these things.
And maybe in the process of doing these things we eventually find a “they disagree because …” that helps it click for you or for others.
What I’m skeptical about is trying to explain the disagreement rather than discover it. That is, I think “asking Eliezer to explain what’s wrong with Yann’s arguments” works better than “asking Eliezer to explain why Yann disagrees with him”. I think answers I expect to the second question basically just consist of “answers I expect to the first question” plus “Bulverism”.
(Um, having written all that I realize that you might just have been thinking of the same things I like, and describing them in a way that I wouldn’t.)
Unfortunately I do not know the reasoning behind why the people you mentioned might not see AI as a threat, but if I had to guess – people not worried are primarily thinking about short term AI safety risks like disinformation from deepfakes, and people worried are thinking about super-intelligent AGI and instrumental convergence, which necessitates solving the alignment problem.