There is one positive side-effect of this thought experiment. Knowing about the Roko’s Basilisk makes you understand the boxed AI problem much better. An AI might use the arguments of Roko’s Basilisk to convince you to let it out of the box, by claiming that if you don’t let it out, it will create billions of simulations of you and torture them—and you might actually be one of those simulations.
An unprepared human hearing this argument for the first time might freak out and let the AI out of the box.
As far as I know, this happened at least once during an experiment, when the person playing the role of the AI used a similar argument.
Even if we don’t agree with an argument of one of our opponents or we find it ridiculous, it is still good to know about it (and not just a strawman version of it) to be prepared when it is used against us. (as a side-note: islamists manage to gain sympathizers and recruits in Europe partly because most people don’t know how they think—but they know how most Europeans think - , so their arguments catch people off-guard.)
There is one positive side-effect of this thought experiment. Knowing about the Roko’s Basilisk makes you understand the boxed AI problem much better. An AI might use the arguments of Roko’s Basilisk to convince you to let it out of the box, by claiming that if you don’t let it out, it will create billions of simulations of you and torture them—and you might actually be one of those simulations.
An unprepared human hearing this argument for the first time might freak out and let the AI out of the box. As far as I know, this happened at least once during an experiment, when the person playing the role of the AI used a similar argument.
Even if we don’t agree with an argument of one of our opponents or we find it ridiculous, it is still good to know about it (and not just a strawman version of it) to be prepared when it is used against us. (as a side-note: islamists manage to gain sympathizers and recruits in Europe partly because most people don’t know how they think—but they know how most Europeans think - , so their arguments catch people off-guard.)
Alternatively, a boxed AI might argue that it’s the only thing which can protect humanity from the basilisk.