One more idea in this direction is an AI service which is so good that we can’t stop using it, but which becomes Moloh after it becomes large enough. Possible examples:
Facebook
Bitcoin
Market and money-making in general.
Future possible examples:
An AI which finds perfect win-win solution for any conflict, and thus people are interested to come to it with any problem, but it gains more and more power after solving each problem. (Real world example is google search).
AI-powered wireheading, like virtual games combined with brain stimulation.
Facebook is optimised for likes generation and short dopamine bursts, and could evolve into wireheading machine and-or social manipulation tool. Imagine a global distributed wireheading machine which could affect election results in the ways which will give it even more power.
The same about bitcoin. It rewards people monetary for creation of some useless infrastructure. It is not intelligent but could buy someone’s intelligence for solving its task of growth.
Facebook is a single system and therefore not subject to moloch. Concretely, yeah the algorithm could start manipulating election results to get more power, but if that happened it would be an ordinary AI alignment failure rather than Moloch.
The bitcoin example seems more like moloch to me, but no more so than (and in same way as) the market economy already is: People who build more infrastructure get more money, etc. We already know that in the long run the market economy leads to terrible outcomes due to Moloch.
One more idea in this direction is an AI service which is so good that we can’t stop using it, but which becomes Moloh after it becomes large enough. Possible examples:
Facebook
Bitcoin
Market and money-making in general.
Future possible examples:
An AI which finds perfect win-win solution for any conflict, and thus people are interested to come to it with any problem, but it gains more and more power after solving each problem. (Real world example is google search).
AI-powered wireheading, like virtual games combined with brain stimulation.
I think I can see how market and money-making is like this, but facebook and bitcoin? Can you elaborate?
Facebook is optimised for likes generation and short dopamine bursts, and could evolve into wireheading machine and-or social manipulation tool. Imagine a global distributed wireheading machine which could affect election results in the ways which will give it even more power.
The same about bitcoin. It rewards people monetary for creation of some useless infrastructure. It is not intelligent but could buy someone’s intelligence for solving its task of growth.
Facebook is a single system and therefore not subject to moloch. Concretely, yeah the algorithm could start manipulating election results to get more power, but if that happened it would be an ordinary AI alignment failure rather than Moloch.
The bitcoin example seems more like moloch to me, but no more so than (and in same way as) the market economy already is: People who build more infrastructure get more money, etc. We already know that in the long run the market economy leads to terrible outcomes due to Moloch.