As far as I can tell, the standard rebuttal to Roko’s idea is based on the strategy of simply ignoring acausal blackmail. But what if you consider the hypothetical situation that there is no blackmail and acausal deals involved, and the Basilisk scenario is how things just are? Simply put, if we accept the Computational Theory of Mind, we need to accept that our experiences could in principle be a simulation by another actor. Then, if you ever find yourself entertaining the question:
Q1: Should I do everything I can to ensure the creation/simulation of an observer that is asking itself the same question as I am now, and will suffer a horrible fate if it answers no?
Would the answer that maximizes your utility not be yes? The idea being that answering no opens up the possibility that you are an actor created by someone who answered yes, and thus, might end up suffering whatever horrible fate he might have implemented. This seems to be the argument that Roko put forth in his original post, and that I have not seen soundly refuted.
Of course, one could argue that there is no evidence for being in such a simulation instead of the real world, and thus that the commitment and ethical dilemma of answering yes is not worth it. If we assign equal probability to all worlds we might inhabit, this might indeed be convincing. However, I think a kind of anthropic argument, based on Bostroms Self-Sampling Assumption (SSA) could be constructed to show that we may indeed be more likely to be in such a simulation. Assume your your hypotheses to be
H0: “I am living in the real world, which is as it appears to be, i.e. the possible observers are the ≈ 7 billion humans on planet earth”.
H1: “The real world is dominated by actors who answered yes to Q1, and (possibly with the help of AGI) filled the universe with simulations of observers pondering Q1”.
Then, given the data
D: “I am an observer thinking about Q1”
And assuming that in the world as it appears to be in H0, only very few people, say, one in ten thousand, asked themselves this question Q1, while under H1, the universe is literally filled with observers thinking about Q1, we get
Pr[D|H0] = 0.0001
Pr[D|H1] ≈ 1
So, the Bayes factor would show very clear evidence in favor of H1. Of course, this would apply to any hypothetical model in which most observers would have exactly the same thoughts as we (and may indeed lead to solipsism), but H1 at least gives a convincing reason on why this should be the case.
So how can we avoid biting the bullet of answering yes to Q1,which seems like a very unattractive option, but possibly still better than being the only actor to answer no? Admittedly, I am quite new to the idea of anthropic reasoning, so my logic could be flawed. I would like to hear thoughts on this.
(Why) Does the Basilisk Argument fail?
As far as I can tell, the standard rebuttal to Roko’s idea is based on the strategy of simply ignoring acausal blackmail. But what if you consider the hypothetical situation that there is no blackmail and acausal deals involved, and the Basilisk scenario is how things just are? Simply put, if we accept the Computational Theory of Mind, we need to accept that our experiences could in principle be a simulation by another actor. Then, if you ever find yourself entertaining the question:
Would the answer that maximizes your utility not be yes? The idea being that answering no opens up the possibility that you are an actor created by someone who answered yes, and thus, might end up suffering whatever horrible fate he might have implemented. This seems to be the argument that Roko put forth in his original post, and that I have not seen soundly refuted.
Of course, one could argue that there is no evidence for being in such a simulation instead of the real world, and thus that the commitment and ethical dilemma of answering yes is not worth it. If we assign equal probability to all worlds we might inhabit, this might indeed be convincing. However, I think a kind of anthropic argument, based on Bostroms Self-Sampling Assumption (SSA) could be constructed to show that we may indeed be more likely to be in such a simulation. Assume your your hypotheses to be
H0: “I am living in the real world, which is as it appears to be, i.e. the possible observers are the ≈ 7 billion humans on planet earth”.
H1: “The real world is dominated by actors who answered yes to Q1, and (possibly with the help of AGI) filled the universe with simulations of observers pondering Q1”.
Then, given the data
D: “I am an observer thinking about Q1”
And assuming that in the world as it appears to be in H0, only very few people, say, one in ten thousand, asked themselves this question Q1, while under H1, the universe is literally filled with observers thinking about Q1, we get
Pr[D|H0] = 0.0001
Pr[D|H1] ≈ 1
So, the Bayes factor would show very clear evidence in favor of H1. Of course, this would apply to any hypothetical model in which most observers would have exactly the same thoughts as we (and may indeed lead to solipsism), but H1 at least gives a convincing reason on why this should be the case.
So how can we avoid biting the bullet of answering yes to Q1, which seems like a very unattractive option, but possibly still better than being the only actor to answer no? Admittedly, I am quite new to the idea of anthropic reasoning, so my logic could be flawed. I would like to hear thoughts on this.