Maybe. I’m not sure about that though. MIRI is the only person or organization I’m aware of that seems to want to create a world controlling AI; and it’s the world-controlling part that I find especially dangerous. That could send MIRI’s AI in directions others won’t go. Are there other organizations attempting to develop AIs to control the world? Is anyone else trying to build a benevolent dictator?
The Machine Intelligence Research Institute exists to ensure that the creation of smarter-than-human intelligence benefits society.
They are well aware of the dangers of creating a uFAI, and you can be certain they will be real careful before they push a button that have the slightest chance of launching the ultimate ending (good or bad). Even then, they may very well decide that “being real careful” is not enough.
Are there other organizations attempting to develop AIs to control the world?
Anthropomorphic ideas of a “robot rebellion,” in which AIs spontaneously develop primate-like resentments of low tribal status, are the stuff of science fiction. The more plausible danger stems not from malice, but from the fact that human survival requires scarce resources: resources for which AIs may have other uses.
Many AIs will converge toward being optimizing systems, in the sense that, after self-modification, they will act to maximize some goal. For instance, AIs developed under evolutionary pressures would be selected for values that maximized reproductive fitness, and would prefer to allocate resources to reproduction rather than supporting humans.
Are there other organizations attempting to develop AIs to control the world? Is anyone else trying to build a benevolent dictator?
Is MIRI attempting to develop any sort of AI? I understood the current focus of its research to be the logic of Friendly AGI, i.e. given the ability to create a superintelligent entity, how do you build one that we would like to have created? This need not involve working on developing one.
Maybe. I’m not sure about that though. MIRI is the only person or organization I’m aware of that seems to want to create a world controlling AI; and it’s the world-controlling part that I find especially dangerous. That could send MIRI’s AI in directions others won’t go. Are there other organizations attempting to develop AIs to control the world? Is anyone else trying to build a benevolent dictator?
MIRI’s stated goal is more meta:
They are well aware of the dangers of creating a uFAI, and you can be certain they will be real careful before they push a button that have the slightest chance of launching the ultimate ending (good or bad). Even then, they may very well decide that “being real careful” is not enough.
It probably doesn’t matter, as any uFAI is likely to emerge by mistake:
Is MIRI attempting to develop any sort of AI? I understood the current focus of its research to be the logic of Friendly AGI, i.e. given the ability to create a superintelligent entity, how do you build one that we would like to have created? This need not involve working on developing one.