MIRI folks are the most prominent proponents of fast takeoff, and we unfortunately haven’t had time to write up a thorough response. Oli already quoted the quick comments I posted from Nate and Eliezer last year, and I’ll chime in with some of the factors that I think are leading to disagreements about takeoff:
Some MIRI people (Nate is one) suspect we might already be in hardware overhang mode, or closer to that point than some other researchers in the field believe.
MIRI folks tend to have different views from Paul about AGI, some of which imply that AGI is more likely to be novel and dependent on new insights. (Unfair caricature: Imagine two people in the early 20th century who don’t have a technical understanding of nuclear physics yet, trying to argue about how powerful a nuclear-chain-reaction-based bomb might be. If one side were to model that kind of bomb as “sort of like TNT 3.0” while the other is modeling it as “sort of like a small Sun”, they’re likely to disagree about whether nuclear weapons are going to be a small v. large improvement over TNT. Note I’m just using nuclear weapons as an analogy, not giving an outside-view argument “sometimes technologies are discontinuous, ergo AGI will be discontinuous”.)
This list isn’t at all intended to be sufficiently-detailed or exhaustive.
I’m hoping we have time to write up more thoughts on this before too long, because this is an important issue (even given that we’re trying to minimize the researcher time we put into things other than object-level deconfusion research). I don’t want MIRI to be a blocker on other researchers making progress on these issues, though — it would be bad if people put a pause on hashing out takeoff issues for themselves (or put a pause on alignment research that’s related to takeoff views) until Eliezer had time to put out a blog post. I primarily wanted to make sure people know that the lack of a substantive response doesn’t mean that Nate+Eliezer+Benya+etc. agree with Paul on takeoff issues now, or that we don’t think this disagreement matters. Our tardiness is because of opportunity costs and because our views have a lot of pieces to articulate.
Rob, any updates on this, e.g., has a longer reply been published somewhere since you wrote this comment, or are you still hoping “we have time to write up more thoughts on this before too long”?
MIRI folks are the most prominent proponents of fast takeoff, and we unfortunately haven’t had time to write up a thorough response. Oli already quoted the quick comments I posted from Nate and Eliezer last year, and I’ll chime in with some of the factors that I think are leading to disagreements about takeoff:
Some MIRI people (Nate is one) suspect we might already be in hardware overhang mode, or closer to that point than some other researchers in the field believe.
MIRI folks tend to have different views from Paul about AGI, some of which imply that AGI is more likely to be novel and dependent on new insights. (Unfair caricature: Imagine two people in the early 20th century who don’t have a technical understanding of nuclear physics yet, trying to argue about how powerful a nuclear-chain-reaction-based bomb might be. If one side were to model that kind of bomb as “sort of like TNT 3.0” while the other is modeling it as “sort of like a small Sun”, they’re likely to disagree about whether nuclear weapons are going to be a small v. large improvement over TNT. Note I’m just using nuclear weapons as an analogy, not giving an outside-view argument “sometimes technologies are discontinuous, ergo AGI will be discontinuous”.)
This list isn’t at all intended to be sufficiently-detailed or exhaustive.
I’m hoping we have time to write up more thoughts on this before too long, because this is an important issue (even given that we’re trying to minimize the researcher time we put into things other than object-level deconfusion research). I don’t want MIRI to be a blocker on other researchers making progress on these issues, though — it would be bad if people put a pause on hashing out takeoff issues for themselves (or put a pause on alignment research that’s related to takeoff views) until Eliezer had time to put out a blog post. I primarily wanted to make sure people know that the lack of a substantive response doesn’t mean that Nate+Eliezer+Benya+etc. agree with Paul on takeoff issues now, or that we don’t think this disagreement matters. Our tardiness is because of opportunity costs and because our views have a lot of pieces to articulate.
Rob, any updates on this, e.g., has a longer reply been published somewhere since you wrote this comment, or are you still hoping “we have time to write up more thoughts on this before too long”?
EY now has longer replies here; https://www.lesswrong.com/posts/vwLxd6hhFvPbvKmBH/yudkowsky-and-christiano-discuss-takeoff-speeds
No long-form reply yet, unfortunately!