You are 200% right. This is the problem we have to solve, not making sure a superintelligent AI can be technologically instructed to serve the whims of its creators.
Have you read Scott Alexander’s Meditations on Moloch? It’s brilliant, and is quite adjacent to the claims you are making. It has received too little follow-up in this community.
The implicit question is – if everyone hates the current system, who perpetuates it? And Ginsberg answers: “Moloch”. It’s powerful not because it’s correct – nobody literally thinks an ancient Carthaginian demon causes everything – but because thinking of the system as an agent throws into relief the degree to which the system isn’t an agent.
(...)
I know that “capitalists sometimes do bad things” isn’t exactly an original talking point. But I do want to stress how it’s not equivalent to “capitalists are greedy”. I mean, sometimes they are greedy. But other times they’re just in a sufficiently intense competition where anyone who doesn’t do it will be outcompeted and replaced by people who do. Business practices are set by Moloch, no one else has any choice in the matter.
(...)
But the current rulers of the universe – call them what you want, Moloch, Gnon, whatever – want us dead, and with us everything we value. Art, science, love, philosophy, consciousness itself, the entire bundle. And since I’m not down with that plan, I think defeating them and taking their place is a pretty high priority.
The opposite of a trap is a garden. The only way to avoid having all human values gradually ground down by optimization-competition is to install a Gardener over the entire universe who optimizes for human values.
(...)
The Universe is a dark and foreboding place, suspended between alien deities. Cthulhu, Gnon, Moloch, call them what you will.
Somewhere in this darkness is another god. He has also had many names. In the Kushiel books, his name was Elua. He is the god of flowers and free love and all soft and fragile things. Of art and science and philosophy and love. Of niceness, community, and civilization. He is a god of humans.
The other gods sit on their dark thrones and think “Ha ha, a god who doesn’t even control any hell-monsters or command his worshippers to become killing machines. What a weakling! This is going to be so easy!”
But somehow Elua is still here. No one knows exactly how. And the gods who oppose Him tend to find Themselves meeting with a surprising number of unfortunate accidents.
Thanks for quoting the bit about Elua at the end. It is helpful to remember that despite Moloch, et al, humanity has managed some pretty impressive feats, even in the present day.
It’s easy to think that the counterexample of science in earlier posts is something accomplished “Once upon a time in a land far away.”
As a concrete example, I’m quite glad that the highly effective mRNA vaccines (Moderna/Pfizer) exist for the common man. They exist despite things like the FDA, the world of academic publishing, the need to find funding to survive, and so on.
Have you read Scott Alexander’s Meditations on Moloch?
Absolutely!
You might like David Deutsch’s book “The Beginning of Infinity”. I read an early draft of one of the chapters (something about “cultural evolution” I think) several years ago. That got me thinking seriously about all this even moreso than Scott’s brilliant essay.
You are 200% right. This is the problem we have to solve, not making sure a superintelligent AI can be technologically instructed to serve the whims of its creators.
Have you read Scott Alexander’s Meditations on Moloch? It’s brilliant, and is quite adjacent to the claims you are making. It has received too little follow-up in this community.
https://www.lesswrong.com/posts/TxcRbCYHaeL59aY7E/meditations-on-moloch
Thanks for quoting the bit about Elua at the end. It is helpful to remember that despite Moloch, et al, humanity has managed some pretty impressive feats, even in the present day.
It’s easy to think that the counterexample of science in earlier posts is something accomplished “Once upon a time in a land far away.”
As a concrete example, I’m quite glad that the highly effective mRNA vaccines (Moderna/Pfizer) exist for the common man. They exist despite things like the FDA, the world of academic publishing, the need to find funding to survive, and so on.
Absolutely!
You might like David Deutsch’s book “The Beginning of Infinity”. I read an early draft of one of the chapters (something about “cultural evolution” I think) several years ago. That got me thinking seriously about all this even moreso than Scott’s brilliant essay.
Looks interesting, thanks for the recommendation!