Suppose that, sometime this decade, designer superbugs end up collapsing civilization as we know it, but not wiping out the entire species. Things are catastrophic, but enough people possess immunity to any particular act of bioterrorism that humanity ultimately pulls though. However, many nation-states collapse and AI capabilities progress, along with much else, regresses for many decades.
From the perspective of the deep future and the 1050 good lives that might inhabit it, how bad was this dark age? Did this narrowly avert a far greater catastrophe, by destroying a global civilization about to wipe out humanity via unaligned AGI, or did it just delay that destruction a century or so? Does a new dark age put us on a course to values unlike our own, such that even if survivor-humanity later succeed at alignment, much of what we care about is lost?
[Question] Longtermist Consequences of a New Dark Age?
Inspired by a talk I had with James Faville.
Suppose that, sometime this decade, designer superbugs end up collapsing civilization as we know it, but not wiping out the entire species. Things are catastrophic, but enough people possess immunity to any particular act of bioterrorism that humanity ultimately pulls though. However, many nation-states collapse and AI capabilities progress, along with much else, regresses for many decades.
From the perspective of the deep future and the 1050 good lives that might inhabit it, how bad was this dark age? Did this narrowly avert a far greater catastrophe, by destroying a global civilization about to wipe out humanity via unaligned AGI, or did it just delay that destruction a century or so? Does a new dark age put us on a course to values unlike our own, such that even if survivor-humanity later succeed at alignment, much of what we care about is lost?