The reason these events were scary, and subsequent fiction was able to capitalise on that, was that they were near misses. Very near misses. There is already a lot of fiction about various misaligned AIs, but that doesn’t affect people much. So what you seem to be advocating is generating some near misses in order to wake everybody up.
Fear is useful. It would be good to have more of it. The question is how plausible it is to generate situations that are scary enough to be useful, but under enough control to be safe.
The reactor meltdown on a Soviet submarine was not posing an existential threat. In the worst case, it would be a little version of Chernobyl. We might compare it to an AI which causes some serious problems, like a stock market crash, but not existential ones. And the movie is not a threat at all.
”The question is how plausible it is to generate situations that are scary enough to be useful, but under enough control to be safe.” That is a great summary of what I wanted to say!
The reason these events were scary, and subsequent fiction was able to capitalise on that, was that they were near misses. Very near misses. There is already a lot of fiction about various misaligned AIs, but that doesn’t affect people much. So what you seem to be advocating is generating some near misses in order to wake everybody up.
Fear is useful. It would be good to have more of it. The question is how plausible it is to generate situations that are scary enough to be useful, but under enough control to be safe.
The reactor meltdown on a Soviet submarine was not posing an existential threat. In the worst case, it would be a little version of Chernobyl. We might compare it to an AI which causes some serious problems, like a stock market crash, but not existential ones. And the movie is not a threat at all.
”The question is how plausible it is to generate situations that are scary enough to be useful, but under enough control to be safe.”
That is a great summary of what I wanted to say!