Certainly MIRI has written about this, for example see the relevant part of their 2018 update:
The latter scenario is relatively less important in worlds where AGI timelines are short. If current deep learning research is already on the brink of AGI, for example, then it becomes less plausible that the results of MIRI’s deconfusion work could become a relevant influence on AI capabilities research, and most of the potential impact of our work would come from its direct applicability to deep-learning-based systems. While many of us at MIRI believe that short timelines are at least plausible, there is significant uncertainty and disagreement about timelines inside MIRI, and I would not feel comfortable committing to a course of action that is safe only in worlds where timelines are short.
Also see Eliezer’s top-notch piece on timelines, which includes the relevant quote:
Of course, the future is very hard to predict in detail. It’s so hard that not only do I confess my own inability, I make the far stronger positive statement that nobody else can do it either.
Eliezer also updated after losing a bet that AlphaGo would not be able to beat humans so well, which he wrote about in AlphaGo Zero and the Foom Debate. It ends with the line:
I wouldn’t have predicted AlphaGo and lost money betting against the speed of its capability gains, because reality held a more extreme position than I did on the Yudkowsky-Hanson spectrum.
More timeline statements, from Eliezer in March 2016:
That said, timelines are the hardest part of AGI issues to forecast, by which I mean that if you ask me for a specific year, I throw up my hands and say “Not only do I not know, I make the much stronger statement that nobody else has good knowledge either.” Fermi said that positive-net-energy from nuclear power wouldn’t be possible for 50 years, two years before he oversaw the construction of the first pile of uranium bricks to go critical. The way these things work is that they look fifty years off to the slightly skeptical, and ten years later, they still look fifty years off, and then suddenly there’s a breakthrough and they look five years off, at which point they’re actually 2 to 20 years off.
If you hold a gun to my head and say “Infer your probability distribution from your own actions, you self-proclaimed Bayesian” then I think I seem to be planning for a time horizon between 8 and 40 years, but some of that because there’s very little I think I can do in less than 8 years, and, you know, if it takes longer than 40 years there’ll probably be some replanning to do anyway over that time period.
Since [August], senior staff at MIRI have reassessed their views on how far off artificial general intelligence (AGI) is and concluded that shorter timelines are more likely than they were previously thinking. [...]
There’s no consensus among MIRI researchers on how long timelines are, and our aggregated estimate puts medium-to-high probability on scenarios in which the research community hasn’t developed AGI by, e.g., 2035. On average, however, research staff now assign moderately higher probability to AGI’s being developed before 2035 than we did a year or two ago.
I talked to Nate last month and he outlined the same concepts and arguments from Eliezer’s Oct. 2017 There’s No Fire Alarm for AGI (mentioned by Ben above) to describe his current view of timelines, in particular (quoting Eliezer’s post):
History shows that for the general public, and even for scientists not in a key inner circle, and even for scientists in that key circle, it is very often the case that key technological developments still seem decades away, five years before they show up. [...]
And again, that’s not to say that people saying “fifty years” is a certain sign that something is happening in a squash court; they were saying “fifty years” sixty years ago too. It’s saying that anyone who thinks technological timelines are actually forecastable, in advance, by people who are not looped in to the leading project’s progress reports and who don’t share all the best ideas about exactly how to do the thing and how much effort is required for that, is learning the wrong lesson from history. In particular, from reading history books that neatly lay out lines of progress and their visible signs that we all know now were important and evidential. It’s sometimes possible to say useful conditional things about the consequences of the big development whenever it happens, but it’s rarely possible to make confident predictions about the timing of those developments, beyond a one- or two-year horizon. And if you are one of the rare people who can call the timing, if people like that even exist, nobody else knows to pay attention to you and not to the Excited Futurists or Sober Skeptics. [...]
So far as I can presently estimate, now that we’ve had AlphaGo and a couple of other maybe/maybe-not shots across the bow, and seen a huge explosion of effort invested into machine learning and an enormous flood of papers, we are probably going to occupy our present epistemic state until very near the end.
By saying we’re probably going to be in roughly this epistemic state until almost the end, I don’t mean to say we know that AGI is imminent, or that there won’t be important new breakthroughs in AI in the intervening time. I mean that it’s hard to guess how many further insights are needed for AGI, or how long it will take to reach those insights. After the next breakthrough, we still won’t know how many more breakthroughs are needed, leaving us in pretty much the same epistemic state as before. Whatever discoveries and milestones come next, it will probably continue to be hard to guess how many further insights are needed, and timelines will continue to be similarly murky. Maybe researcher enthusiasm and funding will rise further, and we’ll be able to say that timelines are shortening; or maybe we’ll hit another AI winter, and we’ll know that’s a sign indicating that things will take longer than they would otherwise; but we still won’t know how long.
I had already seen all of those quotes/links, all of the quotes/links that Rob Bensinger posts in the sibling comment, as well as this tweet from Eliezer. I asked my question because those public quotes don’t sound like the private information I referred to in my question, and I wanted insight into the discrepancy.
Okay. I was responding to “Is there any truth to these claims?” which sounded like it would be a big shock to discover MIRI/CFAR staff were considering short timelines a lot in their actions, when they’d actually stated it out loud in many places.
While I agree that I’m confused about MIRI/CFAR’s timelines and think that info-cascades around this have likely occurred, I want to mention that the thing you linked to is pretty hyperbolic.
To the best of my understanding, part of why the MIRI leadership (Nate Soares, Eliezer Yudkowsky, and Anna Salamon) have been delusionally spewing nonsense about the destruction of the world within a decade is because they’ve been misled by Dario Amodei, an untrustworthy, blatant status-seeker recently employed at Google Brain. I am unaware of the existence of even a single concrete, object-level reason to believe these claims; I, and many others, suspect that Dario is intentionally embellishing the facts because he revels in attention.
I want to say that I think that Dario is not obviously untrustworthy; I think well of him for being an early EA who put in the work to write up their reasoning about donations (see his extensive writeup on the GiveWell blog from 2009) which I always take as a good sign about someone’s soul; the quote also says there’s no reason or argument to believe in short timelines, but the analyses above in Eliezer’s posts on AlphaGo Zero and Fire Alarm provide plenty of reasons for thinking AI could come within a decade. Don’t forget that Shane Legg, one of the cofounders of DeepMind, has been consistently predicting AGI with 50% probability by 2028 (e.g. he said it here in 2011).
Don’t forget that Shane Legg, one of the cofounders of DeepMind, has been consistently predicting AGI with 50% probability by 2028 (e.g. he said it here in 2011).
Just noting that since then, half the time to 2028 has elapsed. If he’s still giving 50%, that’s kind of surprising.
Why is that surprising? Doesn’t it just mean that the pace of development in the last decade has been approximately equal to the average over Shane_{2011}’s distribution of development speeds?
I don’t think it’s that simple. The uncertainty isn’t just about pace of development but about how much development needs to be done.
But even if it does mean that, would that not be surprising? Perhaps not if he’d originally given a narrow confidence internal, but his 10% estimate was in 2018. For us to be hitting the average precisely enough to not move the 50% estimate much… I haven’t done any arithmetic here, but I think that would be surprising, yeah.
And my sense is that the additional complexity makes it more surprising, not less.
Yes, I agree that the space of things to be uncertain about is multidimensional. We project the uncertainty onto a one-dimensional space parameterized by “probability of <event> by <time>”.
It would be surprising for a sophisticated person to show a market of 49 @ 51 on this event. (Unpacking jargon, showing this market means being willing to buy for 49 or sell at 51 a contract which is worth 100 if the hypothesis is true and 0 if it is false)
(it’s somewhat similar saying that your 2-sigma confidence interval around the “true probability” of the event is 49 to 51. The market language can be interpreted with just decision theory while the confidence interval idea also requires some notion of statistics)
My interpretation of the second-hand evidence about Shane Legg’s opinion suggests that Shane would quote a market like 40 @ 60. (The only thing I know about Shane is that they apparently summarized their belief as 50% a number of years ago and hasn’t publicly changed their opinion since)
Perhaps I’m misinterpreting you, but I feel like this was intended as disagreement? If so, I’d appreciate clarification. It seems basically correct to me, and consistent with what I said previously. I still think that: if, in 2011, you gave 10% probability by 2018 and 50% by 2028; and if, in 2019, you still give 50% by 2028 (as an explicit estimate, i.e. you haven’t just not-given an updated estimate); then this is surprising, even acknowledging that 50% is probably not very precise in either case.
I realised after writing that I didn’t give a quote to show he that still believed it. I have the recollection that he still says 2028, I think someone more connected to AI/ML probably told me, but I can’t think of anywhere to quote him saying it.
Certainly MIRI has written about this, for example see the relevant part of their 2018 update:
Also see Eliezer’s top-notch piece on timelines, which includes the relevant quote:
Eliezer also updated after losing a bet that AlphaGo would not be able to beat humans so well, which he wrote about in AlphaGo Zero and the Foom Debate. It ends with the line:
More timeline statements, from Eliezer in March 2016:
And from me in April 2017:
I talked to Nate last month and he outlined the same concepts and arguments from Eliezer’s Oct. 2017 There’s No Fire Alarm for AGI (mentioned by Ben above) to describe his current view of timelines, in particular (quoting Eliezer’s post):
I had already seen all of those quotes/links, all of the quotes/links that Rob Bensinger posts in the sibling comment, as well as this tweet from Eliezer. I asked my question because those public quotes don’t sound like the private information I referred to in my question, and I wanted insight into the discrepancy.
Okay. I was responding to “Is there any truth to these claims?” which sounded like it would be a big shock to discover MIRI/CFAR staff were considering short timelines a lot in their actions, when they’d actually stated it out loud in many places.
While I agree that I’m confused about MIRI/CFAR’s timelines and think that info-cascades around this have likely occurred, I want to mention that the thing you linked to is pretty hyperbolic.
I want to say that I think that Dario is not obviously untrustworthy; I think well of him for being an early EA who put in the work to write up their reasoning about donations (see his extensive writeup on the GiveWell blog from 2009) which I always take as a good sign about someone’s soul; the quote also says there’s no reason or argument to believe in short timelines, but the analyses above in Eliezer’s posts on AlphaGo Zero and Fire Alarm provide plenty of reasons for thinking AI could come within a decade. Don’t forget that Shane Legg, one of the cofounders of DeepMind, has been consistently predicting AGI with 50% probability by 2028 (e.g. he said it here in 2011).
Just noting that since then, half the time to 2028 has elapsed. If he’s still giving 50%, that’s kind of surprising.
Why is that surprising? Doesn’t it just mean that the pace of development in the last decade has been approximately equal to the average over Shane_{2011}’s distribution of development speeds?
I don’t think it’s that simple. The uncertainty isn’t just about pace of development but about how much development needs to be done.
But even if it does mean that, would that not be surprising? Perhaps not if he’d originally given a narrow confidence internal, but his 10% estimate was in 2018. For us to be hitting the average precisely enough to not move the 50% estimate much… I haven’t done any arithmetic here, but I think that would be surprising, yeah.
And my sense is that the additional complexity makes it more surprising, not less.
Yes, I agree that the space of things to be uncertain about is multidimensional. We project the uncertainty onto a one-dimensional space parameterized by “probability of <event> by <time>”.
It would be surprising for a sophisticated person to show a market of 49 @ 51 on this event. (Unpacking jargon, showing this market means being willing to buy for 49 or sell at 51 a contract which is worth 100 if the hypothesis is true and 0 if it is false)
(it’s somewhat similar saying that your 2-sigma confidence interval around the “true probability” of the event is 49 to 51. The market language can be interpreted with just decision theory while the confidence interval idea also requires some notion of statistics)
My interpretation of the second-hand evidence about Shane Legg’s opinion suggests that Shane would quote a market like 40 @ 60. (The only thing I know about Shane is that they apparently summarized their belief as 50% a number of years ago and hasn’t publicly changed their opinion since)
Perhaps I’m misinterpreting you, but I feel like this was intended as disagreement? If so, I’d appreciate clarification. It seems basically correct to me, and consistent with what I said previously. I still think that: if, in 2011, you gave 10% probability by 2018 and 50% by 2028; and if, in 2019, you still give 50% by 2028 (as an explicit estimate, i.e. you haven’t just not-given an updated estimate); then this is surprising, even acknowledging that 50% is probably not very precise in either case.
I realised after writing that I didn’t give a quote to show he that still believed it. I have the recollection that he still says 2028, I think someone more connected to AI/ML probably told me, but I can’t think of anywhere to quote him saying it.