I largely agree with Rob Bensinger’s comments in his posted Google doc in the comments section of the market.
These are frustratingly non-disjunctive.
Here’s what I think:
Humanity probably needs a delay. Simple alignment methods probably won’t just straightforwardly work. Not that they provably can’t, just that I think they probably won’t. I give that like a 2% chance.
Some fraction of humanity will see the need for delay, and some fraction won’t. There will be significant tensions. I expect that the fraction that does believe in the need for slowdown will undertake some dramatic heroic actions. I expect that takeoff won’t be so sudden (e.g. a few days from foom to doom) that this opportunity will never arise.
If we get lucky, some persuasive-but-not-too-harmful catastrophes will aid in swaying more of humanity towards delay.
We might figure out alignment directly after more research during the delay period. If so, narrow tool AI will probably have been of some assistance.
We might delay long enough that the separate path of human intelligence augmentation creates substantially smarter humans who end up contributing to alignment research.
Hopefully all this will manage to be accomplished without widespread large-scale destruction from nuclear weapons, but if things get really dire it might come to that. It’s weird that I grew up hating and fearing nuclear weapons technology and our MAD standoff, but now I’m grateful we have it as a last ditch out which could plausibly save humanity.
I largely agree with Rob Bensinger’s comments in his posted Google doc in the comments section of the market. These are frustratingly non-disjunctive. Here’s what I think: Humanity probably needs a delay. Simple alignment methods probably won’t just straightforwardly work. Not that they provably can’t, just that I think they probably won’t. I give that like a 2% chance. Some fraction of humanity will see the need for delay, and some fraction won’t. There will be significant tensions. I expect that the fraction that does believe in the need for slowdown will undertake some dramatic heroic actions. I expect that takeoff won’t be so sudden (e.g. a few days from foom to doom) that this opportunity will never arise. If we get lucky, some persuasive-but-not-too-harmful catastrophes will aid in swaying more of humanity towards delay. We might figure out alignment directly after more research during the delay period. If so, narrow tool AI will probably have been of some assistance. We might delay long enough that the separate path of human intelligence augmentation creates substantially smarter humans who end up contributing to alignment research. Hopefully all this will manage to be accomplished without widespread large-scale destruction from nuclear weapons, but if things get really dire it might come to that. It’s weird that I grew up hating and fearing nuclear weapons technology and our MAD standoff, but now I’m grateful we have it as a last ditch out which could plausibly save humanity.