We should increase awareness of old fairy tales with a jinn who misinterprets wishes.
The most popular UFAI story I’m aware of is “The Sorcerer’s Apprentice”.
Sticking with European folktales that were made into classic Disney cartoons, maybe the analogy to be made is “AI isn’t Pinocchio. It’s Mickey’s enchanted brooms. It doesn’t want to be a Real Boy; it just wants to carry water. The danger isn’t that it will grow up to be a naughty boy if it doesn’t listen to its conscience. It’s that it cannot care about anything other than carrying water; including whether or not it’s flooding your home.”
Thing is, much of the popular audience doesn’t really know what code is. They’ve never written a bug and had a program do something unintended … because they’ve never written any code at all. They’ve certainly never written a virus or worm, or even a script that accidentally overwrites their files with zeroes. They may have issued a bad order to a computer (“Oops, I shouldn’t have sent that email!”) but they’ve never composed and run a non-obviously bad set of instructions.
So, aside from folklore, better CS education may be part of the story here.
The most popular UFAI story I’m aware of is “The Sorcerer’s Apprentice”.
Sticking with European folktales that were made into classic Disney cartoons, maybe the analogy to be made is “AI isn’t Pinocchio. It’s Mickey’s enchanted brooms. It doesn’t want to be a Real Boy; it just wants to carry water. The danger isn’t that it will grow up to be a naughty boy if it doesn’t listen to its conscience. It’s that it cannot care about anything other than carrying water; including whether or not it’s flooding your home.”
Thing is, much of the popular audience doesn’t really know what code is. They’ve never written a bug and had a program do something unintended … because they’ve never written any code at all. They’ve certainly never written a virus or worm, or even a script that accidentally overwrites their files with zeroes. They may have issued a bad order to a computer (“Oops, I shouldn’t have sent that email!”) but they’ve never composed and run a non-obviously bad set of instructions.
So, aside from folklore, better CS education may be part of the story here.