I quite agree. I feel rather like humanity is a bunch of children playing with matches in a drought-stricken forest. Sooner or later something’s bound to get out of hand.
My one small point of disagreement here is that I think perhaps we are closer to potentially dangerous human-out-of-the-loop RSI than this post suggests.
My own takeaway from GPT-4 and other recent developments is that we’re more likely to see the kind of smooth, gradual takeoff described by Christiano in e.g. the 2021 MIRI conversations (already seeing it, even?), but at a speed too fast to be useful for making progress on aligning even the gradual-takeoff systems.
And if we don’t survive gradual takeoff, we don’t even get a chance to try surviving hard takeoff, regardless of how close it is. I don’t really have any strong beliefs about how likely or when hard takeoff happens, other than “probably after gradual takeoff, assuming there is an after”, and even that I’m not that confident in.
Yeah, good point. Like the story about the guy who shot himself while jumping off the cliff into the ocean. Gotta dodge the bullet before you worry about the landing.
I quite agree. I feel rather like humanity is a bunch of children playing with matches in a drought-stricken forest. Sooner or later something’s bound to get out of hand. My one small point of disagreement here is that I think perhaps we are closer to potentially dangerous human-out-of-the-loop RSI than this post suggests.
My own takeaway from GPT-4 and other recent developments is that we’re more likely to see the kind of smooth, gradual takeoff described by Christiano in e.g. the 2021 MIRI conversations (already seeing it, even?), but at a speed too fast to be useful for making progress on aligning even the gradual-takeoff systems.
And if we don’t survive gradual takeoff, we don’t even get a chance to try surviving hard takeoff, regardless of how close it is. I don’t really have any strong beliefs about how likely or when hard takeoff happens, other than “probably after gradual takeoff, assuming there is an after”, and even that I’m not that confident in.
Yeah, good point. Like the story about the guy who shot himself while jumping off the cliff into the ocean. Gotta dodge the bullet before you worry about the landing.