Split the cake into parts and provide it to different extrapolated volitions that didn’t cohere, after allocating some defense army patrol keeping the borders from future war?
“Thou O Queen canst scarcely know our grief; for to thee the blessed Gods gave thy heart’s desire: youth for ever, and peace. Would they might give us our good gift, that should be youth for ever, and war; and unwaning strength and skill in arms. Would they might but give us our great enemies alive and whole again. For better it were we should run hazard again of utter destruction, than thus live out our lives like cattle fattening for the slaughter, or like silly garden plants.”
True, but it would nevertheless make for a decent compromise. Do you have a better suggestion?
It isn’t much of a compromise. It presumes enough coherence for everyone to agree on a supreme power that all shall obey, to divide the cake and enforce the borders.
To see how the situation is actually handled, look around you, at the whole world now and in the past.
Whenever there is no common resolve to leave each other alone, then as has been observed of old, the strong do what they will and the weak bear what they must. Europe managed it in the Treaty of Westphalia, but it took thirty years of slaughtering each other for them to decide that no-one was ever going to win, and draw up a massive agreement to disagree. Best of luck getting any such agreement today between (for example) jihadists and everyone else. Or SJWs and neoreactionaries. Adding superintelligence would be like squabbling children getting hold of nuclear weapons.
One faction might have values that lead to something highly dis-valued by another faction (ie. one faction values slavery, another opposes slavery for all beings, even those of the first faction).
allocating some defense army patrol keeping the borders from future war?
Rather than use traditional army methods, it’s probably more efficient to have the SI play the role of Sysop in this scenario, and just deny human actors access to base-layer reality; though if one wanted to allow communication between the different domains, the sysop may still need to run some active defense against high-level information attacks.
If human values are not capable of becoming coherent, and humanity comes to know that, what should be done?
Split the cake into parts and provide it to different extrapolated volitions that didn’t cohere, after allocating some defense army patrol keeping the borders from future war?
That itself would go against some values.
E.R. Eddision, The Worm Ouroboros
True, but it would nevertheless make for a decent compromise. Do you have a better suggestion?
It isn’t much of a compromise. It presumes enough coherence for everyone to agree on a supreme power that all shall obey, to divide the cake and enforce the borders.
To see how the situation is actually handled, look around you, at the whole world now and in the past. Whenever there is no common resolve to leave each other alone, then as has been observed of old, the strong do what they will and the weak bear what they must. Europe managed it in the Treaty of Westphalia, but it took thirty years of slaughtering each other for them to decide that no-one was ever going to win, and draw up a massive agreement to disagree. Best of luck getting any such agreement today between (for example) jihadists and everyone else. Or SJWs and neoreactionaries. Adding superintelligence would be like squabbling children getting hold of nuclear weapons.
One faction might have values that lead to something highly dis-valued by another faction (ie. one faction values slavery, another opposes slavery for all beings, even those of the first faction).
Rather than use traditional army methods, it’s probably more efficient to have the SI play the role of Sysop in this scenario, and just deny human actors access to base-layer reality; though if one wanted to allow communication between the different domains, the sysop may still need to run some active defense against high-level information attacks.