True. Unless there were very good arguments/very good evidence for one side or the other. My expectation is that for any random hypothesis there will be lots of disagreement about it among experts. For a random hypothesis with lots of good arguments/good evidence, I would expect much, much less disagreement among experts in the field.
If we look at climate change, for example, the vast majority of experts agreed about it quite early on—within 15 years of the Charney report.
If all I am left with, however, is ‘smart person believes silly thing for silly reasons’ then it is not reasonable for me as a lay person to determine which is the silly thing. Is ‘AI poses no (or extremely low) x-risk’ the silly thing, or is ‘AI poses unacceptable x-risk’ the silly thing?
If AI does indeed pose unacceptable x-risk and there are good arguments/good evidence for this, then there also has to be a good reason or set of reasons why many experts are not convinced. (Yann claims, for example, that the AI experts arguing for AI x-risk are a very small minority and Eliezer Yudkowsky seems to agree with this).
If we look at climate change, for example, the vast majority of experts agreed about it quite early on—within 15 years of the Charney report.
So I don’t know much about timelines of global warming or global warming science, but I note that that report came out in 1979, more than 100 years after the industrial revolution. So it’s not clear to me that fifteen years after that counts as “quite early on”, or that AI science is currently at a comparable point in the timeline. (If points in these timelines can even be compared.)
If all I am left with, however, is ‘smart person believes silly thing for silly reasons’ then it is not reasonable for me as a lay person to determine which is the silly thing.
FWIW I think even relatively-lay people can often detect silly arguments, even from people who know a lot more than them. Some examples where I think I’ve done that:
I remember seeing someone (possibly even Yann LeCun?) saying something along the lines of, AGI is impossible because of no free lunch theorems.
Someone saying that HPMOR’s “you violated conservation of energy!” bit is dumb because something something quantum stuff that I didn’t understand; and also because if turning into a cat violated conservation of energy, then so did levitating someone a few paragraphs earlier. I am confident this person (who went by the handle su3su2u1)
knows a lot more about physics than me. I am also confident this second part was them being silly.
So I’d suggest that you might be underestimating yourself.
But if you’re right that you can’t reasonably figure this out… I’m not sure there are any ways to get around that? Eliezer can say “Yann believes this because of optimism bias” and Yann can say “Eliezer believes this because of availability heuristic” or whatever, and maybe one or both of them is right (tbc I have not observed either of them saying these things). But these are both Bulverism.
It may be that Eliezer and Yann can find a double crux, something where they agree: “Eliezer believes X, and if Eliezer believed not-X then Eliezer would think AGI does not pose a serious risk. Yann believes not-X, and if Yann believed X then Yann would think AGI does pose a serious risk.” But finding such Xs is hard, I don’t expect there to be a simple one, and even if there was it just punts the question: “why do these two smart people disagree on X?” It’s possible X is in a domain that you consider yourself better able to have an opinion on, but it’s also possible it’s in one you consider yourself less able to have an opinion on.
If AI does indeed pose unacceptable x-risk and there are good arguments/good evidence for this, then there also has to be a good reason or set of reasons why many experts are not convinced.
I basically just don’t think there does have to be this.
(Yann claims, for example, that the AI experts arguing for AI x-risk are a very small minority and Eliezer Yudkowsky seems to agree with this)
Fwiw my sense is that this is false, and that Yann might believe it but I don’t expect Eliezer to. But I don’t remember what I’ve seen that makes me think this. (To some extent it might depend on who you count as an expert and what you count as arguing for x-risk.)
Re timelines for climate change, in the 1970s, serious people in the field of climate studies started suggesting that there was a serious problem looming. A very short time later, the entire field was convinced by the evidence and argument for that serious risk—to the point that the IPCC was established in 1988 by the UN.
When did some serious AI researchers start to suggest that there was a serious problem looming? I think in the 2000s. There is no IPAIX-risk.
And, yes: I can detect silly arguments in a reasonable number of cases. But I have not been able to do so in this case as yet (in the aggregate). It seems that there are possibly good arguments on both sides.
It is indeed tricky—I also mentioned that it could get into a regress-like situation. But I think that if people like me are to be convinced it might be worth the attempt. As you say, there may be a more accessible to me domain in there somewhere.
Re the numbers, Eliezer seems to claim that the majority of AI researchers believe in X-risk, but few are speaking out for a variety of reasons. This boils down to me trusting Eliezer’s word about the majority belief, because that majority is not speaking out. He may be motivated to lie in this case—note that I am not saying that he is, but ‘lying for Jesus’ (for example) is a relatively common thing. It is also possible that he is not lying but is wrong—he may have talked to a sample that was biased in some way.
Re timelines for climate change, in the 1970s, serious people in the field of climate studies started suggesting that there was a serious problem looming. A very short time later, the entire field was convinced by the evidence and argument for that serious risk—to the point that the IPCC was established in 1988 by the UN.
When did some serious AI researchers start to suggest that there was a serious problem looming? I think in the 2000s. There is no IPAIX-risk.
Nod. But then, I assume by the 1970s there was already observable evidence of warming? Whereas the observable evidence of AI X-risk in the 2000s seems slim. Like I expect I could tell a story for global warming along the lines of “some people produced a graph with a trend line, and some people came up with theories to explain it”, and for AI X-risk I don’t think we have graphs or trend lines of the same quality.
This isn’t particularly a crux for me btw. But like, there are similarities and differences between these two things, and pointing out the similarities doesn’t really make me expect that looking at one will tell us much about the other.
I think that if people like me are to be convinced it might be worth the attempt. As you say, there may be a more accessible to me domain in there somewhere.
Not opposed to trying, but like...
So I think it’s basically just good to try to explain things more clearly and to try to get to the roots of disagreements. There are lots of ways this can look like. We can imagine a conversation between Eliezer and Yann, or people who respectively agree with them. We can imagine someone currently unconvinced having individual conversations with each side. We can imagine discussions playing out through essays written over the course of months. We can imagine FAQs written by each side which give their answers to the common objections raised by the other. I like all these things.
And maybe in the process of doing these things we eventually find a “they disagree because …” that helps it click for you or for others.
What I’m skeptical about is trying to explain the disagreement rather than discover it. That is, I think “asking Eliezer to explain what’s wrong with Yann’s arguments” works better than “asking Eliezer to explain why Yann disagrees with him”. I think answers I expect to the second question basically just consist of “answers I expect to the first question” plus “Bulverism”.
(Um, having written all that I realize that you might just have been thinking of the same things I like, and describing them in a way that I wouldn’t.)
True. Unless there were very good arguments/very good evidence for one side or the other. My expectation is that for any random hypothesis there will be lots of disagreement about it among experts. For a random hypothesis with lots of good arguments/good evidence, I would expect much, much less disagreement among experts in the field.
If we look at climate change, for example, the vast majority of experts agreed about it quite early on—within 15 years of the Charney report.
If all I am left with, however, is ‘smart person believes silly thing for silly reasons’ then it is not reasonable for me as a lay person to determine which is the silly thing. Is ‘AI poses no (or extremely low) x-risk’ the silly thing, or is ‘AI poses unacceptable x-risk’ the silly thing?
If AI does indeed pose unacceptable x-risk and there are good arguments/good evidence for this, then there also has to be a good reason or set of reasons why many experts are not convinced. (Yann claims, for example, that the AI experts arguing for AI x-risk are a very small minority and Eliezer Yudkowsky seems to agree with this).
So I don’t know much about timelines of global warming or global warming science, but I note that that report came out in 1979, more than 100 years after the industrial revolution. So it’s not clear to me that fifteen years after that counts as “quite early on”, or that AI science is currently at a comparable point in the timeline. (If points in these timelines can even be compared.)
FWIW I think even relatively-lay people can often detect silly arguments, even from people who know a lot more than them. Some examples where I think I’ve done that:
I remember seeing someone (possibly even Yann LeCun?) saying something along the lines of, AGI is impossible because of no free lunch theorems.
Someone saying that HPMOR’s “you violated conservation of energy!” bit is dumb because something something quantum stuff that I didn’t understand; and also because if turning into a cat violated conservation of energy, then so did levitating someone a few paragraphs earlier. I am confident this person (who went by the handle su3su2u1) knows a lot more about physics than me. I am also confident this second part was them being silly.
This comment.
So I’d suggest that you might be underestimating yourself.
But if you’re right that you can’t reasonably figure this out… I’m not sure there are any ways to get around that? Eliezer can say “Yann believes this because of optimism bias” and Yann can say “Eliezer believes this because of availability heuristic” or whatever, and maybe one or both of them is right (tbc I have not observed either of them saying these things). But these are both Bulverism.
It may be that Eliezer and Yann can find a double crux, something where they agree: “Eliezer believes X, and if Eliezer believed not-X then Eliezer would think AGI does not pose a serious risk. Yann believes not-X, and if Yann believed X then Yann would think AGI does pose a serious risk.” But finding such Xs is hard, I don’t expect there to be a simple one, and even if there was it just punts the question: “why do these two smart people disagree on X?” It’s possible X is in a domain that you consider yourself better able to have an opinion on, but it’s also possible it’s in one you consider yourself less able to have an opinion on.
I basically just don’t think there does have to be this.
Fwiw my sense is that this is false, and that Yann might believe it but I don’t expect Eliezer to. But I don’t remember what I’ve seen that makes me think this. (To some extent it might depend on who you count as an expert and what you count as arguing for x-risk.)
Re timelines for climate change, in the 1970s, serious people in the field of climate studies started suggesting that there was a serious problem looming. A very short time later, the entire field was convinced by the evidence and argument for that serious risk—to the point that the IPCC was established in 1988 by the UN.
When did some serious AI researchers start to suggest that there was a serious problem looming? I think in the 2000s. There is no IPAIX-risk.
And, yes: I can detect silly arguments in a reasonable number of cases. But I have not been able to do so in this case as yet (in the aggregate). It seems that there are possibly good arguments on both sides.
It is indeed tricky—I also mentioned that it could get into a regress-like situation. But I think that if people like me are to be convinced it might be worth the attempt. As you say, there may be a more accessible to me domain in there somewhere.
Re the numbers, Eliezer seems to claim that the majority of AI researchers believe in X-risk, but few are speaking out for a variety of reasons. This boils down to me trusting Eliezer’s word about the majority belief, because that majority is not speaking out. He may be motivated to lie in this case—note that I am not saying that he is, but ‘lying for Jesus’ (for example) is a relatively common thing. It is also possible that he is not lying but is wrong—he may have talked to a sample that was biased in some way.
Nod. But then, I assume by the 1970s there was already observable evidence of warming? Whereas the observable evidence of AI X-risk in the 2000s seems slim. Like I expect I could tell a story for global warming along the lines of “some people produced a graph with a trend line, and some people came up with theories to explain it”, and for AI X-risk I don’t think we have graphs or trend lines of the same quality.
This isn’t particularly a crux for me btw. But like, there are similarities and differences between these two things, and pointing out the similarities doesn’t really make me expect that looking at one will tell us much about the other.
Not opposed to trying, but like...
So I think it’s basically just good to try to explain things more clearly and to try to get to the roots of disagreements. There are lots of ways this can look like. We can imagine a conversation between Eliezer and Yann, or people who respectively agree with them. We can imagine someone currently unconvinced having individual conversations with each side. We can imagine discussions playing out through essays written over the course of months. We can imagine FAQs written by each side which give their answers to the common objections raised by the other. I like all these things.
And maybe in the process of doing these things we eventually find a “they disagree because …” that helps it click for you or for others.
What I’m skeptical about is trying to explain the disagreement rather than discover it. That is, I think “asking Eliezer to explain what’s wrong with Yann’s arguments” works better than “asking Eliezer to explain why Yann disagrees with him”. I think answers I expect to the second question basically just consist of “answers I expect to the first question” plus “Bulverism”.
(Um, having written all that I realize that you might just have been thinking of the same things I like, and describing them in a way that I wouldn’t.)