When you hear about “the dangers of AI”, what do you think of? Probably a bad actor using AI to hurt others, or a sci-fi scenario of robots turning evil. However, the bigger harm is more likely to be misalignment: an AI smarter than humans, without sharing human values. The top research labs, at places like DeepMind and OpenAI, are working to create superhuman AI, yet the current paradigm trains AI with simple goals. Detecting faces, trading stocks, maximizing some metric or other. So if super-intelligent AI is invented, it will probably seek to fulfill a narrow, parochial goal. With its mental capacity, faster speed, and ability to copy itself, it could take powerful actions to reach its goal. Since human values, wants, and needs are complicated and poorly-understood, the AI is unlikely to care about anyone who gets in its way. It doesn’t turn evil, it simply bulldozes through humans who aren’t smart enough to fight it. Without safeguards, we could end up like the ants whose hills we pave over with our highways. (Policymakers)
When you hear about “the dangers of AI”, what do you think of? Probably a bad actor using AI to hurt others, or a sci-fi scenario of robots turning evil. However, the bigger harm is more likely to be misalignment: an AI smarter than humans, without sharing human values. The top research labs, at places like DeepMind and OpenAI, are working to create superhuman AI, yet the current paradigm trains AI with simple goals. Detecting faces, trading stocks, maximizing some metric or other. So if super-intelligent AI is invented, it will probably seek to fulfill a narrow, parochial goal. With its mental capacity, faster speed, and ability to copy itself, it could take powerful actions to reach its goal. Since human values, wants, and needs are complicated and poorly-understood, the AI is unlikely to care about anyone who gets in its way. It doesn’t turn evil, it simply bulldozes through humans who aren’t smart enough to fight it. Without safeguards, we could end up like the ants whose hills we pave over with our highways. (Policymakers)