I think there’s actually a ton of uncertainty here about just how ‘exploitable’ human civilization ultimately is. We could imagine that since actual humans (e.g. Hitler) by talking to people have seized large fractions of Earth’s resources, we might not need an AI that’s all that much smarter than a human. On the other hand, we might just say that attempts like that are filtered through colossal amounts of luck and historical contingency and actually to reliably manipulate your way to controlling most of humanity you’d need to be far smarter than the smartest human.
What about the current situation in Russia? I think Putin must be winging the propaganda effort, since he wasn’t expecting to have to fight a long and hard war, plus some of the messaging don’t stand up to even cursory inspection (a Jewish Nazi president?), and yet it’s still working remarkably well.
Putin is already president of Russia. The steps between an AI being president of a country and killing everybody is pretty cut-and-dry; I could probably do that if I had an AI’s value function. The steps between an AI being a computer program assigned to raise the stock price of GOOG$ and consistently becoming president of Russia are much less clear.
My point here was that humans seem so susceptible to propaganda that an AI can probably find some way to bend us to its will. But if you want a more specific strategy that the AI could use (which I think came to me in part because I had the current war in the back of my mind), see my top level comment here.
The Putin case would be better if he was convincing Russians to make massive sacrifices or do something that will backfire and kill them, like start a war with NATO, and I don’t think he has that power—e.g. him rushing to deny that Russia were sending conscripts to Ukraine because of the fear the effect that would have on public opinion
Yeah, but Putin’s been president of Russia for over 20 years and already has a very large, loyal following. There will always be those that enthusiastically follow the party line of the leader. It’s somewhat harder to actually seize power. (None of this is to excuse the actions of Putin or those who support him.)
I think there’s actually a ton of uncertainty here about just how ‘exploitable’ human civilization ultimately is. We could imagine that since actual humans (e.g. Hitler) by talking to people have seized large fractions of Earth’s resources, we might not need an AI that’s all that much smarter than a human. On the other hand, we might just say that attempts like that are filtered through colossal amounts of luck and historical contingency and actually to reliably manipulate your way to controlling most of humanity you’d need to be far smarter than the smartest human.
What about the current situation in Russia? I think Putin must be winging the propaganda effort, since he wasn’t expecting to have to fight a long and hard war, plus some of the messaging don’t stand up to even cursory inspection (a Jewish Nazi president?), and yet it’s still working remarkably well.
Putin is already president of Russia. The steps between an AI being president of a country and killing everybody is pretty cut-and-dry; I could probably do that if I had an AI’s value function. The steps between an AI being a computer program assigned to raise the stock price of GOOG$ and consistently becoming president of Russia are much less clear.
My point here was that humans seem so susceptible to propaganda that an AI can probably find some way to bend us to its will. But if you want a more specific strategy that the AI could use (which I think came to me in part because I had the current war in the back of my mind), see my top level comment here.
The Putin case would be better if he was convincing Russians to make massive sacrifices or do something that will backfire and kill them, like start a war with NATO, and I don’t think he has that power—e.g. him rushing to deny that Russia were sending conscripts to Ukraine because of the fear the effect that would have on public opinion
Yeah, but Putin’s been president of Russia for over 20 years and already has a very large, loyal following. There will always be those that enthusiastically follow the party line of the leader. It’s somewhat harder to actually seize power. (None of this is to excuse the actions of Putin or those who support him.)