Cameron, I suppose that’s a fair enough comment. I’m used to the way things work in AI, where the naive simply fail utterly and completely to accomplish anything whatsoever, rather than hurting anyone else or themselves, and you have to get pretty far to get beyond that to the realm of dangers.
Not to mention, I’m used to the idiom that the lack of any prior feedback or any second try is what makes something an adult problem—but that really is representative of the dangers faced by someone able to modify their own brain circuitry. If there’s an AI that can say “No” but you’re allowed to ignore the “No” then one mistake is fatal. The sort of people who think, “Gee, I’ll just run myself with the modification for a while and see what happens ’cuz I can always go back”—they might ignore a “No” based on their concept of “testing”, and then that would be the end of them.
You want to put the dangerous things behind a challenge/lock such that by the time you pass it you know how dangerous they really are. “Make an AI”, unfortunately, may not be quite strong enough as a case of this, but “Make an AI without anyone helping you on a 100MHz computer with 1GB of RAM and 100GB of disk space” is probably strong enough.
Cameron, I suppose that’s a fair enough comment. I’m used to the way things work in AI, where the naive simply fail utterly and completely to accomplish anything whatsoever, rather than hurting anyone else or themselves, and you have to get pretty far to get beyond that to the realm of dangers.
Not to mention, I’m used to the idiom that the lack of any prior feedback or any second try is what makes something an adult problem—but that really is representative of the dangers faced by someone able to modify their own brain circuitry. If there’s an AI that can say “No” but you’re allowed to ignore the “No” then one mistake is fatal. The sort of people who think, “Gee, I’ll just run myself with the modification for a while and see what happens ’cuz I can always go back”—they might ignore a “No” based on their concept of “testing”, and then that would be the end of them.
You want to put the dangerous things behind a challenge/lock such that by the time you pass it you know how dangerous they really are. “Make an AI”, unfortunately, may not be quite strong enough as a case of this, but “Make an AI without anyone helping you on a 100MHz computer with 1GB of RAM and 100GB of disk space” is probably strong enough.