Singleton solutions—there will be no coordination problems if everything is ruled by one royal dynasty / one political party / one recursively self-improving artificial intelligence.
Royal dynasties and political parties are not Singletons by any stretch of the imagination. Infighting is Moloch. But even if we assumed an immortal benevolent human dictator, a dictator only exercises power through keys to power and still has to constantly fight off competition for his power. Stalin didn’t start the Great Purge for shits and giggles; it’s a tried and true strategy used by rulers throughout history.
The hope with artificial superintelligence is that, due to the wide design space of possible AIs, we can perhaps pick one that is sub-agent stable and free of mesa-optimization, and also more powerful than all other agents in the universe combined by a huge margin. If no AI can satisfy these conditions, we are just as doomed.
Primitivism solutions—all problems will be simple if we make our lifestyle simple.
That’s not defeating Moloch, that’s surrendering completely and unconditionally to Moloch in its original form of natural selection.
An immortal benevolent human dictator isn’t a singleton either. Human cells tend to cooperate to make humans because it tends to be their most effective competitive strategy. The cells in an immortal all powerful human dictator would have a different payoff matrix and would likely start defecting over time.
Royal dynasties and political parties are not Singletons by any stretch of the imagination. Infighting is Moloch. But even if we assumed an immortal benevolent human dictator, a dictator only exercises power through keys to power and still has to constantly fight off competition for his power. Stalin didn’t start the Great Purge for shits and giggles; it’s a tried and true strategy used by rulers throughout history.
The hope with artificial superintelligence is that, due to the wide design space of possible AIs, we can perhaps pick one that is sub-agent stable and free of mesa-optimization, and also more powerful than all other agents in the universe combined by a huge margin. If no AI can satisfy these conditions, we are just as doomed.
That’s not defeating Moloch, that’s surrendering completely and unconditionally to Moloch in its original form of natural selection.
An immortal benevolent human dictator isn’t a singleton either. Human cells tend to cooperate to make humans because it tends to be their most effective competitive strategy. The cells in an immortal all powerful human dictator would have a different payoff matrix and would likely start defecting over time.