It seems that @Scott Alexander believes that there’s a 50%+ chance we all die
It’s “we end up dead or careening towards Venezuela” in the original, which is not the same thing. Venezuela has survivors. Existence of survivors is the crucial distinction between extinction and global catastrophe. AGI would be a much more reasonable issue if it was merely risking global catastrophe.
In the first couple sentences he says “if we never get AI, I expect the future to be short and grim. Most likely we kill ourselves with synthetic biology.” So it seems he’s putting most of his probability mass on everyone dying.
But then after he says: “But if we ban all gameboard-flipping technologies, then we do end up with bioweapon catastrophe or social collapse.”
I think people who responding are seemingly only reading the Venezuela part and assuming most of the probability mass he’s putting in the 50% is just a ‘catastrophe’ like Venezuela. But then why would he say he expects the future to be short conditional on no AI?
It’s a bit ambiguous, but “bioweapon catastrophe or social collapse” is not literal extinction, and I’m reading “I expect the future to be short and grim” as plausibly referring to destruction of uninterrupted global civilization, which might well recover after 3000 years. The text doesn’t seem to rule out this interpretation.
Sufficiently serious synthetic biology catastrophes prevent more serious further catastrophes, including by destroying civilization, and it’s not very likely that this involves literal extinction. As a casual reader of his blogs over the years, I’m not aware of Scott’s statements to the effect that his position is different from this, either clearly stated or in aggregate from many vague claims.
It’s “we end up dead or careening towards Venezuela” in the original, which is not the same thing. Venezuela has survivors. Existence of survivors is the crucial distinction between extinction and global catastrophe. AGI would be a much more reasonable issue if it was merely risking global catastrophe.
In the first couple sentences he says “if we never get AI, I expect the future to be short and grim. Most likely we kill ourselves with synthetic biology.” So it seems he’s putting most of his probability mass on everyone dying.
But then after he says: “But if we ban all gameboard-flipping technologies, then we do end up with bioweapon catastrophe or social collapse.”
I think people who responding are seemingly only reading the Venezuela part and assuming most of the probability mass he’s putting in the 50% is just a ‘catastrophe’ like Venezuela. But then why would he say he expects the future to be short conditional on no AI?
It’s a bit ambiguous, but “bioweapon catastrophe or social collapse” is not literal extinction, and I’m reading “I expect the future to be short and grim” as plausibly referring to destruction of uninterrupted global civilization, which might well recover after 3000 years. The text doesn’t seem to rule out this interpretation.
Sufficiently serious synthetic biology catastrophes prevent more serious further catastrophes, including by destroying civilization, and it’s not very likely that this involves literal extinction. As a casual reader of his blogs over the years, I’m not aware of Scott’s statements to the effect that his position is different from this, either clearly stated or in aggregate from many vague claims.