I’m like 96% sure it was intended to apply to the question of how much of the work in making an AGI is about “cultural general-intelligence software”. But yeah, I agree that if we destroy our civilization it could take a long time to get it back. Not just because building a civilization takes a long time; also because there are various resources we’ve probably consumed most of the most accessible bits of, and not having such easy access to coal and oil and minerals could make building a new civilization much harder. But I’m not sure what hangs on that (as opposed to the related but separate question of whether we would rebuild civilization if we lost it) -- the destruction of human civilization would be a calamity, but I’m not sure it would be a much worse calamity if it took 300k years to repair than if it took “only” 30k years.
I think it matters because of what it implies about how hard a target civilization is to reach. Even if the 300k year process could be sped up a lot by knowing what we’re aiming for, it’s evidence that the end result was a much weaker natural attractor than our current state is, from a starting point of founding civilization at all.
I’m like 96% sure it was intended to apply to the question of how much of the work in making an AGI is about “cultural general-intelligence software”. But yeah, I agree that if we destroy our civilization it could take a long time to get it back. Not just because building a civilization takes a long time; also because there are various resources we’ve probably consumed most of the most accessible bits of, and not having such easy access to coal and oil and minerals could make building a new civilization much harder. But I’m not sure what hangs on that (as opposed to the related but separate question of whether we would rebuild civilization if we lost it) -- the destruction of human civilization would be a calamity, but I’m not sure it would be a much worse calamity if it took 300k years to repair than if it took “only” 30k years.
I think it matters because of what it implies about how hard a target civilization is to reach. Even if the 300k year process could be sped up a lot by knowing what we’re aiming for, it’s evidence that the end result was a much weaker natural attractor than our current state is, from a starting point of founding civilization at all.