Bureaucracy of AIs: This runs into serious anthropomorphism problems. Bureaucracies might be spectacularly skilled at insulating itself against all sorts of rich, brilliant, and/or powerful outsiders, and it might also be true that this evolved through survival of the fittest and trial and error.
However, a major factor in suppressing influence and halting change is the constant omnipresence of aging. Power struggles are not competitive evolution or an efficient market for ambitious people; they are an organizational hemorrhaging that routinely happens whenever a senior manager begins to go senile, and loses the faculties necessary to fire/depose anyone who poses the slightest threat of overthrowing them (and gradually loses those faculties too).
Instead of an efficient market of ambitious power seekers, the ensuing power struggle is dominated by slightly-less-senior managers who happened to be in the right place in the right time, and found themselves thrust into an opportunity to become the alpha male. Some of them have been dreaming of that for decades, others haven’t, but all of them are old and wise enough to wonder what power is worth to them and how much risk they’re willing to take on in order to get it. Uncertainty, emotion, and life-changing decisions abound, and mistakes and failures are commonplace in the resulting maneuverings.
The only way to get AI to replicate that reliably is with a heavily humanlike mind, and adheres to human failures too consistently to do any of the really smart thinking that AGI is built for.
TL;DR I don’t know much about game theory or multi-AGI systems/simulations, but bureaucratic management is too heavily predicated on every moving part being 1) similarly intelligent and 2) organizational atrophy due to powerful people slowly going senile. Bureaucracy might have some potential for some inspiration for multi-AI alignment systems that detect and punish innovation and superior intelligence, but not very much inspiration beyond a handful of undetectable springtraps that let weaker AIs triumph over smarter AIs.
These all sound like really important questions that we should be dedicating a ton of effort/resources into researching. Especially since there is a 50% chance we will discover immortality this century and a 30% chance we will do so before discovering AGI.
Bureaucracy of AIs: This runs into serious anthropomorphism problems. Bureaucracies might be spectacularly skilled at insulating itself against all sorts of rich, brilliant, and/or powerful outsiders, and it might also be true that this evolved through survival of the fittest and trial and error.
However, a major factor in suppressing influence and halting change is the constant omnipresence of aging. Power struggles are not competitive evolution or an efficient market for ambitious people; they are an organizational hemorrhaging that routinely happens whenever a senior manager begins to go senile, and loses the faculties necessary to fire/depose anyone who poses the slightest threat of overthrowing them (and gradually loses those faculties too).
Instead of an efficient market of ambitious power seekers, the ensuing power struggle is dominated by slightly-less-senior managers who happened to be in the right place in the right time, and found themselves thrust into an opportunity to become the alpha male. Some of them have been dreaming of that for decades, others haven’t, but all of them are old and wise enough to wonder what power is worth to them and how much risk they’re willing to take on in order to get it. Uncertainty, emotion, and life-changing decisions abound, and mistakes and failures are commonplace in the resulting maneuverings.
The only way to get AI to replicate that reliably is with a heavily humanlike mind, and adheres to human failures too consistently to do any of the really smart thinking that AGI is built for.
TL;DR I don’t know much about game theory or multi-AGI systems/simulations, but bureaucratic management is too heavily predicated on every moving part being 1) similarly intelligent and 2) organizational atrophy due to powerful people slowly going senile. Bureaucracy might have some potential for some inspiration for multi-AI alignment systems that detect and punish innovation and superior intelligence, but not very much inspiration beyond a handful of undetectable springtraps that let weaker AIs triumph over smarter AIs.
These all sound like really important questions that we should be dedicating a ton of effort/resources into researching. Especially since there is a 50% chance we will discover immortality this century and a 30% chance we will do so before discovering AGI.