It’s interesting, but it’s also, as far as I can tell, wrong.
Birch is willing to concede that if I know that almost all humans live in a simulation, and I know nothing else that would help me distinguish myself from an average human, then I should be almost certain that I’m living in a simulation; i.e., P(I live in a simulation | almost everybody lives in a simulation) ~ 1. More generally, he’s willing to accept that P(I live in a simulation | a fraction x of all humans live in a simulation) = x; similar to how, if I know that 60% of all humans have a gene that has no observable effects, and I don’t know anything about whether I specifically have that gene, I should assign 60% probability to the proposition that I have that gene.
However, Bostrom’s argument rests on the idea that our physics experiments show that there is a lot of computational power in the universe that can in principle be used for simulations. Birch points out that if we live in a simulation, then our physics experiments don’t necessarily give good information about the true computational power in the universe. My first intuition would be that the argument still goes through if we don’t live in a simulation, so perhaps we can derive an almost-contradiction from that? [ETA: Hm, that wasn’t a very good explanation; Eliezer’s comment does better.] Birch considers such a variation and concludes that we would need a principle that P(I live in a simulation | if I don’t live in a simulation, then a fraction x of all humans lives in a simulation) >= x, and he doesn’t see a compelling reason to believe that. (The if-then is a logical implication.)
But this follows from the principle he’s willing to accept. “If I don’t live in a simulation, then a fraction x of all humans lives in a simulation” is logically equivalent to (A or B), where A = “A fraction x of all humans lives in a simulation” and B = “the fraction of all humans that live in a simulation is != x, but I, in particular, live in a simulation”; note that A and B are mutually exclusive. Birch is willing to accept that P(I live in a simulation | A) = x, and it’s certainly true that P(I live in a simulation | B) = 1. Writing p := P(A | A or B), we get
P(SIM | A or B) = [P(SIM | A) p] + [P(SIM | B) (1-p)] = [x * p] + [1-p] >= x.
Thanks Benja. This is a good objection to the argument I make in the ‘Rejecting Good Evidence’ section of the paper, but I think I can avoid it by formulating BIP* more carefully.
Suppose I’m in a situation in which it currently appears to me as though f-sim = x. In effect, your suggestion is that, in this situation, my evidence can be characterized by the disjunction (A ∨ B). You then reason as follows:
(1) Conditional on A, my credence in SIM should be >= x.
(2) Conditional on B, my credence in SIM should be 1.
(3) So overall, given that A and B are mutually exclusive, my credence in
SIM should be >= x.
I accept that this is a valid argument. The problem with it, in my view, is that (A ∨ B) is not a complete description of what my evidence says.
Let V represent the proposition that my evidence regarding f-sim is veridical (i.e., the true value of f-sim is indeed what it appears to be). If A is true, then V is also true. So a more complete description of what my evidence says is (A ∧ V) ∨ (B ∧ ~V).
Now we need to ask: is it true that, conditional on (A ∧ V), my credence in SIM should be >= x?
BIP doesn’t entail that it should be, since BIP takes no account of the relevance of V. And V is surely relevant, since (on the face of it, at least) V is far more likely to be true if I am not simulated (i.e., Cr (V | ~SIM) >> Cr (V | SIM)).
Indeed, if one were to learn that (A ∧ V) is true, one might well rationally assign credence =< x to SIM. However, it’s not important to my argument that one’s credence should be =< x: all that matters is that there is no compelling reason to think that it should be >= x.
In short, then, your argument shows that, conditional on a certain description of what my evidence indicates, my credence in SIM should be >= x. But that description is the not the most complete description available—and we must always use the most complete description available, because we often find in epistemology that incomplete descriptions of the evidence lead to incorrect inferences.
Nevertheless, I think your response does expose an error in the paper. I should have formulated BIP* like this, explicitly introducing V:
When BIP* is formulated like this, it is not entailed by BIP. Yet this is the modified principle Bostrom actually needs, if he wants to recover his original conclusion while rejecting Good Evidence. So I think the overall argument still stands, once the error you point out is corrected.
Well—look, you can’t possibly fix your argument by reformulating BIP*, because your paper gives a correct mathematical proof that its version of BIP*, plus the “quadripartite disjunction”, are enough to imply the simulation argument! :-)
(For people who haven’t read the paper: the quadripartite disjunction is H1 = almost no human civilizations survive to posthumanOR H2 = almost no posthuman civilizations run ancestor simulationsOR H3 = almost all humans live in a simulationOR SIM = I live in a simulation. To get the right intuition, note that this is logically equivalent to “If I live in the real world, i.e. if not SIM, then H1 or H2 or H3”.)
More formally, the argument in your paper shows that BIP* plus Cr(quadripartite disjunction) ~ 1 implies Cr(SIM) >~ 1 - Cr(H1 or H2).
I think that (a) there’s a confusion about what the symbol Cr(.) means, and (b) what you’re really trying to do is to deny Bostrom’s original BIP.
Your credence symbol must be meant to already condition on all your information; recall that the question your paper is examining is whether we should accept that Cr(SIM) ~ 1, which is only an interesting question if this is supposed to take into account our current information. A conditional credence, like Cr(SIM | A), must thus mean: If I knew all I know now, plus the one additional fact that A is true, what would my credence be that I live in a simulation?
[ETA: I.e., the stuff we’re conditioning on is not supposed to represent our state of knowledge, it is hypothetical propositions we’re taking into account in addition to our knowledge! The reason I’m interested in the BIP* from the paper is not that I consider (A or B) a good representation of our state of knowledge (in which case Cr(SIM | A or B) would simply be equal to Cr(SIM)); rather, the reason is that the argument in your paper shows that together with the quadripartite disjunction it is sufficient to give the simulation argument.]
So Bostrom’s BIP, which reads Cr(SIM | f_sim = x) = x, means that given all your current information, if you knew the one additional fact that the fraction of humans that live in a simulation is x, then your credence in yourself living in a simulation would be x. If you want to argue that the simulation argument fails even if our current evidence supports the quadripartite disjunction, because the fact that we observe what we observe gives us additional information that we need to take into account, then you need to argue that BIP is false. I can see ways in which you could try to do this: For example, you could question whether the particle accelerators of simulated humans would reliably work in accordance with quantum mechanics, and if one doesn’t believe this, then we have additional information suggesting we’re in the outside world. More generally, you’d have to identify something that we observe that none (or very very close to none) of the simulated humans would. A very obvious variant of the DNA analogy illustrates this: If the gene gives you black hair, and you have red hair, then being told that 60% of all humans have the gene shouldn’t make you assign a 60% probability to having the gene.
The obvious way to take this into account in the formal argument would be to redefine f_sim to refer to, instead of all humans, only to those humans that live in simulations in which physics etc. looks pretty much like in the outside world; i.e., f_sim says how many of those humans actually live in a simulation. Then, the version of BIP referring to this new f_sim should be uncontroversial, and the above counterarguments would become an attack on the quadripartite disjunction (which is sensible, because they’re arguments about the world, and the quadripartite disjunction is where all the empirically motivated input to the argument is supposed to go).
H2 = almost no posthuman civilizations run ancestor simulations
isn’t nearly specific enough to serve the purposes of the simulation argument. We need to say something about what kinds of evidence would appear to the simulations.
If your case is that BIP is insufficient to establish the conclusions Bostrom wants to establish, I’m pretty sure it does in fact suffice. If you accept both of these:
BIP: Cr[SIM|f-sim ≥ x] ≥ x (where f-sim is over all observers in our evidential situation)
Cr[f-sim ≥ x |V ] Cr[V|¬SIM] ≥ y_x
then we derive Cr[SIM] ≥ 1- (1-x)/y_x. x is some estimate of what f-sim might be in our world if we are not in a simulation and our current evidence is veridical, and y_x is our estimate of how likely a large f-sim is given the same assumptions; it’s likely to be around f_I f_p.
I think your interpretation of “if I don’t live in a simulation, then a fraction x of all humans lives in a simulation” as P(SIM or A) is wrong; it makes more sense to interpret it as P(A|¬SIM). This actually makes the proof simpler: for any A, B, we have that P(A) ≤ P(A|B)/P(B|A) by Bayes theorem, so if we accept that P(¬SIM|A) = (1-x), then we have P(¬SIM) ≤ (1-x)/P(A|¬SIM).
I think your interpretation of “if I don’t live in a simulation, then a fraction x of all humans lives in a simulation” as P(SIM or A) is wrong
Huh?
The paper talks about P(SIM | ¬SIM → A), which is equal to P(SIM | SIM ∨ A) because ¬SIM → A is logically equivalent to SIM ∨ A. I wrote the P(SIM | ¬SIM → A) from the paper in words as P(I live in a simulation | if I don’t live in a simulation, then a fraction x of all humans lives in a simulation) and stated explicitly that the if-then was a logical implication. I didn’t talk about P(SIM or A) anywhere.
Mark Eichenlaub and Patrick LaVictoire point out on Facebook that if we let p=P(A|B) and q=P(B|A), there’s a bound P(A) ⇐ p/(p+q-pq), which is smaller than p/q.
It’s interesting, but it’s also, as far as I can tell, wrong.
Birch is willing to concede that if I know that almost all humans live in a simulation, and I know nothing else that would help me distinguish myself from an average human, then I should be almost certain that I’m living in a simulation; i.e., P(I live in a simulation | almost everybody lives in a simulation) ~ 1. More generally, he’s willing to accept that P(I live in a simulation | a fraction x of all humans live in a simulation) = x; similar to how, if I know that 60% of all humans have a gene that has no observable effects, and I don’t know anything about whether I specifically have that gene, I should assign 60% probability to the proposition that I have that gene.
However, Bostrom’s argument rests on the idea that our physics experiments show that there is a lot of computational power in the universe that can in principle be used for simulations. Birch points out that if we live in a simulation, then our physics experiments don’t necessarily give good information about the true computational power in the universe. My first intuition would be that the argument still goes through if we don’t live in a simulation, so perhaps we can derive an almost-contradiction from that? [ETA: Hm, that wasn’t a very good explanation; Eliezer’s comment does better.] Birch considers such a variation and concludes that we would need a principle that P(I live in a simulation | if I don’t live in a simulation, then a fraction x of all humans lives in a simulation) >= x, and he doesn’t see a compelling reason to believe that. (The if-then is a logical implication.)
But this follows from the principle he’s willing to accept. “If I don’t live in a simulation, then a fraction x of all humans lives in a simulation” is logically equivalent to (A or B), where A = “A fraction x of all humans lives in a simulation” and B = “the fraction of all humans that live in a simulation is != x, but I, in particular, live in a simulation”; note that A and B are mutually exclusive. Birch is willing to accept that P(I live in a simulation | A) = x, and it’s certainly true that P(I live in a simulation | B) = 1. Writing p := P(A | A or B), we get
P(SIM | A or B) = [P(SIM | A) p] + [P(SIM | B) (1-p)] = [x * p] + [1-p] >= x.
Thanks Benja. This is a good objection to the argument I make in the ‘Rejecting Good Evidence’ section of the paper, but I think I can avoid it by formulating BIP* more carefully.
Suppose I’m in a situation in which it currently appears to me as though f-sim = x. In effect, your suggestion is that, in this situation, my evidence can be characterized by the disjunction (A ∨ B). You then reason as follows:
(1) Conditional on A, my credence in SIM should be >= x.
(2) Conditional on B, my credence in SIM should be 1.
(3) So overall, given that A and B are mutually exclusive, my credence in SIM should be >= x.
I accept that this is a valid argument. The problem with it, in my view, is that (A ∨ B) is not a complete description of what my evidence says.
Let V represent the proposition that my evidence regarding f-sim is veridical (i.e., the true value of f-sim is indeed what it appears to be). If A is true, then V is also true. So a more complete description of what my evidence says is (A ∧ V) ∨ (B ∧ ~V).
Now we need to ask: is it true that, conditional on (A ∧ V), my credence in SIM should be >= x?
BIP doesn’t entail that it should be, since BIP takes no account of the relevance of V. And V is surely relevant, since (on the face of it, at least) V is far more likely to be true if I am not simulated (i.e., Cr (V | ~SIM) >> Cr (V | SIM)).
Indeed, if one were to learn that (A ∧ V) is true, one might well rationally assign credence =< x to SIM. However, it’s not important to my argument that one’s credence should be =< x: all that matters is that there is no compelling reason to think that it should be >= x.
In short, then, your argument shows that, conditional on a certain description of what my evidence indicates, my credence in SIM should be >= x. But that description is the not the most complete description available—and we must always use the most complete description available, because we often find in epistemology that incomplete descriptions of the evidence lead to incorrect inferences.
Nevertheless, I think your response does expose an error in the paper. I should have formulated BIP* like this, explicitly introducing V:
BIP*: Cr [SIM | ((f-sim = x) ∧ V) ∨ ((f-sim ≠ x) ∧ ~V)] >= x
When BIP* is formulated like this, it is not entailed by BIP. Yet this is the modified principle Bostrom actually needs, if he wants to recover his original conclusion while rejecting Good Evidence. So I think the overall argument still stands, once the error you point out is corrected.
Well—look, you can’t possibly fix your argument by reformulating BIP*, because your paper gives a correct mathematical proof that its version of BIP*, plus the “quadripartite disjunction”, are enough to imply the simulation argument! :-)
(For people who haven’t read the paper: the quadripartite disjunction is H1 = almost no human civilizations survive to posthuman OR H2 = almost no posthuman civilizations run ancestor simulations OR H3 = almost all humans live in a simulation OR SIM = I live in a simulation. To get the right intuition, note that this is logically equivalent to “If I live in the real world, i.e. if not SIM, then H1 or H2 or H3”.)
More formally, the argument in your paper shows that BIP* plus Cr(quadripartite disjunction) ~ 1 implies Cr(SIM) >~ 1 - Cr(H1 or H2).
I think that (a) there’s a confusion about what the symbol Cr(.) means, and (b) what you’re really trying to do is to deny Bostrom’s original BIP.
Your credence symbol must be meant to already condition on all your information; recall that the question your paper is examining is whether we should accept that Cr(SIM) ~ 1, which is only an interesting question if this is supposed to take into account our current information. A conditional credence, like Cr(SIM | A), must thus mean: If I knew all I know now, plus the one additional fact that A is true, what would my credence be that I live in a simulation?
[ETA: I.e., the stuff we’re conditioning on is not supposed to represent our state of knowledge, it is hypothetical propositions we’re taking into account in addition to our knowledge! The reason I’m interested in the BIP* from the paper is not that I consider (A or B) a good representation of our state of knowledge (in which case Cr(SIM | A or B) would simply be equal to Cr(SIM)); rather, the reason is that the argument in your paper shows that together with the quadripartite disjunction it is sufficient to give the simulation argument.]
So Bostrom’s BIP, which reads Cr(SIM | f_sim = x) = x, means that given all your current information, if you knew the one additional fact that the fraction of humans that live in a simulation is x, then your credence in yourself living in a simulation would be x. If you want to argue that the simulation argument fails even if our current evidence supports the quadripartite disjunction, because the fact that we observe what we observe gives us additional information that we need to take into account, then you need to argue that BIP is false. I can see ways in which you could try to do this: For example, you could question whether the particle accelerators of simulated humans would reliably work in accordance with quantum mechanics, and if one doesn’t believe this, then we have additional information suggesting we’re in the outside world. More generally, you’d have to identify something that we observe that none (or very very close to none) of the simulated humans would. A very obvious variant of the DNA analogy illustrates this: If the gene gives you black hair, and you have red hair, then being told that 60% of all humans have the gene shouldn’t make you assign a 60% probability to having the gene.
The obvious way to take this into account in the formal argument would be to redefine f_sim to refer to, instead of all humans, only to those humans that live in simulations in which physics etc. looks pretty much like in the outside world; i.e., f_sim says how many of those humans actually live in a simulation. Then, the version of BIP referring to this new f_sim should be uncontroversial, and the above counterarguments would become an attack on the quadripartite disjunction (which is sensible, because they’re arguments about the world, and the quadripartite disjunction is where all the empirically motivated input to the argument is supposed to go).
isn’t nearly specific enough to serve the purposes of the simulation argument. We need to say something about what kinds of evidence would appear to the simulations.
If your case is that BIP is insufficient to establish the conclusions Bostrom wants to establish, I’m pretty sure it does in fact suffice. If you accept both of these:
BIP: Cr[SIM|f-sim ≥ x] ≥ x (where f-sim is over all observers in our evidential situation)
Cr[f-sim ≥ x |V ] Cr[V|¬SIM] ≥ y_x
then we derive Cr[SIM] ≥ 1- (1-x)/y_x. x is some estimate of what f-sim might be in our world if we are not in a simulation and our current evidence is veridical, and y_x is our estimate of how likely a large f-sim is given the same assumptions; it’s likely to be around f_I f_p.
(Updated)
I think your interpretation of “if I don’t live in a simulation, then a fraction x of all humans lives in a simulation” as P(SIM or A) is wrong; it makes more sense to interpret it as P(A|¬SIM). This actually makes the proof simpler: for any A, B, we have that P(A) ≤ P(A|B)/P(B|A) by Bayes theorem, so if we accept that P(¬SIM|A) = (1-x), then we have P(¬SIM) ≤ (1-x)/P(A|¬SIM).
Huh?
The paper talks about P(SIM | ¬SIM → A), which is equal to P(SIM | SIM ∨ A) because ¬SIM → A is logically equivalent to SIM ∨ A. I wrote the P(SIM | ¬SIM → A) from the paper in words as P(I live in a simulation | if I don’t live in a simulation, then a fraction x of all humans lives in a simulation) and stated explicitly that the if-then was a logical implication. I didn’t talk about P(SIM or A) anywhere.
Mark Eichenlaub and Patrick LaVictoire point out on Facebook that if we let p=P(A|B) and q=P(B|A), there’s a bound P(A) ⇐ p/(p+q-pq), which is smaller than p/q.
I’ve only skimmed the paper but AFAICT this wipes out its central claim. Good work.