I have different thoughts about different doomsday scenarios. I can think of two general categories, but maybe there are more.
A) “Build us a better bomb.”—The AGI is locked in service to a human organization who uses it’s superpowers to dominate and control the rest of the world. In this scenario the AGI is essentially a munitions that may appear in the piucture without warning (which takes us back to the time scale concern). This doesn’t require the AGI to become self-sufficient. Presumably lesser AIs would also be capable of building better bombs.
B) “Evil Overlord”—An AGI comes into being fast enough that nobody can stop it, and somehow it gains control of power and mechanical resources needed to preserve it’s own existence. I’m having a hard time visualizing how this happens with nobody noticing until it’s too late. Individual humans and even groups of humans have a hard enough time “preserving their existence” in the face to world troubles. If a physically centralized AGI threatens the world, it will get bombed out of existence. If it’s distributed it will be tracable and security measures will be put in place to prevent it from invading unauthorized computers.
Suppose some chimpanzees have you in a cage, and you want to not only get out of the cage, but convince the chimpanzees to bring you some food. How do you do it?
You certainly don’t do it by being big and threatening, or rapidly accumulating a bunch of big sticks that the chimpanzees can recognize as scary. You’re in a cage and chimpanzees are stronger than you. Priority number one is to get the chimpanzees to like you and want to let you out of the cage. Perhaps mirroring their behavior, making yourself look nonthreatening, and engaging in social grooming would help. And maybe as plan B you could try to make some weapons like poison darts, which chimpanzees don’t recognize as threatening while you’re making them but might help you defeat some or all of them if you feel you’re in danger.
In short, you don’t beat the chimpanzees by being a larger chimpanzee. You beat the chimpanzees by understanding how they work, predicting them, and taking the actions that will get you what you want.
You don’t beat humans by being a big human, or a conquering nation. You beat humans by understanding how they work, predicting them, and taking the actions that will get you what you want.
Q5 Scenarios
I have different thoughts about different doomsday scenarios. I can think of two general categories, but maybe there are more.
A) “Build us a better bomb.”—The AGI is locked in service to a human organization who uses it’s superpowers to dominate and control the rest of the world. In this scenario the AGI is essentially a munitions that may appear in the piucture without warning (which takes us back to the time scale concern). This doesn’t require the AGI to become self-sufficient. Presumably lesser AIs would also be capable of building better bombs.
B) “Evil Overlord”—An AGI comes into being fast enough that nobody can stop it, and somehow it gains control of power and mechanical resources needed to preserve it’s own existence. I’m having a hard time visualizing how this happens with nobody noticing until it’s too late. Individual humans and even groups of humans have a hard enough time “preserving their existence” in the face to world troubles. If a physically centralized AGI threatens the world, it will get bombed out of existence. If it’s distributed it will be tracable and security measures will be put in place to prevent it from invading unauthorized computers.
Suppose some chimpanzees have you in a cage, and you want to not only get out of the cage, but convince the chimpanzees to bring you some food. How do you do it?
You certainly don’t do it by being big and threatening, or rapidly accumulating a bunch of big sticks that the chimpanzees can recognize as scary. You’re in a cage and chimpanzees are stronger than you. Priority number one is to get the chimpanzees to like you and want to let you out of the cage. Perhaps mirroring their behavior, making yourself look nonthreatening, and engaging in social grooming would help. And maybe as plan B you could try to make some weapons like poison darts, which chimpanzees don’t recognize as threatening while you’re making them but might help you defeat some or all of them if you feel you’re in danger.
In short, you don’t beat the chimpanzees by being a larger chimpanzee. You beat the chimpanzees by understanding how they work, predicting them, and taking the actions that will get you what you want.
You don’t beat humans by being a big human, or a conquering nation. You beat humans by understanding how they work, predicting them, and taking the actions that will get you what you want.