humans won’t be able to design and enforce a better equilibrium before it’s too late
Won’t the uploaded humans be able to do this? If you think the current world isn’t already on an irreversible course towards unacceptable value drift(such that a singleton is needed to repair things) I don’t see how the mass upload scenario is much worse, since any agreement we could negotiate to mitigate drift, the uploads could make too. The uploads would now have the ability to copy themselves and run at different speeds, but that doesn’t seem to obviously make coordination much harder.
The problem is that willingness to change gives power, leaving those most concerned with current value drift behind, hence “evolution”. With enough change, original stakeholders become disempowered, and additionally interventions that suffice to stop or reverse this process become more costly.
So interventions are more plausible to succeed if done in advance, before the current equilibrium is unmoored. Which requires apparently superhuman foresight.
Won’t the uploaded humans be able to do this? If you think the current world isn’t already on an irreversible course towards unacceptable value drift(such that a singleton is needed to repair things) I don’t see how the mass upload scenario is much worse, since any agreement we could negotiate to mitigate drift, the uploads could make too. The uploads would now have the ability to copy themselves and run at different speeds, but that doesn’t seem to obviously make coordination much harder.
The problem is that willingness to change gives power, leaving those most concerned with current value drift behind, hence “evolution”. With enough change, original stakeholders become disempowered, and additionally interventions that suffice to stop or reverse this process become more costly.
So interventions are more plausible to succeed if done in advance, before the current equilibrium is unmoored. Which requires apparently superhuman foresight.