If we build an AI that’s as smart as a human, that’s not a human. There will be all sorts of things we did wrong, including deliberately, in order to optimize for performance. All someone has to do is crank it up 10x faster in order to get slightly better results, and that could be it. Something smarter than any human, thinking hundreds or millions of times faster than a human mind, able to learn or figure out anything it wants, including how to outmaneuver all human creators.
Techxecutives and ML researchers. I’d worry about sending this to policymakers, because bureaucracies are structured to create unsolvable mazes in order to remain stable, no matter how many rich, intelligent opportunists try to compromise it from the outside.
If we build an AI that’s as smart as a human, that’s not a human. There will be all sorts of things we did wrong, including deliberately, in order to optimize for performance. All someone has to do is crank it up 10x faster in order to get slightly better results, and that could be it. Something smarter than any human, thinking hundreds or millions of times faster than a human mind, able to learn or figure out anything it wants, including how to outmaneuver all human creators.
Techxecutives and ML researchers. I’d worry about sending this to policymakers, because bureaucracies are structured to create unsolvable mazes in order to remain stable, no matter how many rich, intelligent opportunists try to compromise it from the outside.