It’s not obvious to me that this scenario concentrates net probability mass onto ‘things go awesome for humanity long-term’. Making everything harder might mean that alignment is also harder. A few extra years of chaos doesn’t buy us anything unless we’re actively nailing down useful robust AGI during that time.
(There is some extra hope in ‘For some reason, humanity has working AGIs for a little while before anyone can destroy the world, and this doesn’t make alignment much harder’, though I’d assume there are other, much larger contributors-of-hope in any world like that where things actually go well.)
It’s not obvious to me that this scenario concentrates net probability mass onto ‘things go awesome for humanity long-term’. Making everything harder might mean that alignment is also harder. A few extra years of chaos doesn’t buy us anything unless we’re actively nailing down useful robust AGI during that time.
(There is some extra hope in ‘For some reason, humanity has working AGIs for a little while before anyone can destroy the world, and this doesn’t make alignment much harder’, though I’d assume there are other, much larger contributors-of-hope in any world like that where things actually go well.)