I believe two things about rulers (politicians, CEOs of big orgs):
They only give others as much freedom as necessary to be useful to achieve ruler’s goals
They don’t want actors more powerful than them anywhere nearby
From these I intuit that:
Rulers will not support development of powerful AGI as it might threaten to overpower them
Rulers might get rid of humans as soon as an AI can achieve goals more efficiently (but that’s much lower bar for intelligence and power of AI, than that needed to overpower the Ruler)
Thus my immediate fears are not so much about aligning super-human AGI, but about aligning Rulers with needs of their constituents—for example a future in which we never get to smarter than humans AIs, but things a bit more powerful than Office365 Copilot can be sufficient for CEO (or real stakeholders behind) to run the whole company, or for an autocratic president to run enough of the industry to make her a yacht and some caviar.
Question: are any of my two assumptions or two intuitions or the conclusion wrong?
What are some falsifiable, observable predictions of them, which I could verify using internet today?
is probably true, but only because you used the word “powerful” rather than “capable”. Rulers would definitely want development of capable AGIs as long as they believe (however incorrectly) in their ability to maintain power/control over those AGIs.
In fact, rulers are likely to be particularly good at cultivating capable underlings they they maintain firm control of. It may cause them to overestimate their ability to do the same for AGI. In fact, if they expect an AGI to be less agentic, they might expect it to actually be easier to maintain control over an “we just program it to obey” AGI, and prefer that over what they perceive to be inherently less predictable humans.
ChatGPT’s answer:
(I am a bit worried by this given that China seems to restrict AIs more than US...)
I like how ChatGPT can help in operatinalizing fuzzy intuitions. I feel an eerie risk that it makes me think even less, and less carefully, and defer to AIs wisdom more and more… it’s very tempting … as if finding an adult who you can cede control to.