To be clear, after I made it, I thought more about it and I’m not sure it’s correct. I think I’d have to actually do the math, my intuitions aren’t coming in loud and clear here. The reason I’m unsure is that even if for some reason post-apocalyptic worlds rarely get simulated (and thus it’s very unsurprising that we find ourself in a world that didn’t suffer an apocalypse, because we’re probably in a simulation) it may be that we ought to ignore this, since we are trying to act as if we are not simulated anyway, since that’s how we have the most influence or something.
if for some reason post-apocalyptic worlds rarely get simulated
To draw out the argument a little further, the reason that post-apocalyptic worlds don’t get simulated is because most (?) of the simulations of our era are a way to simulate super intelligences in other parts of the multiverse, to talk or trade with.
(As in the basic argument of this Jan Tallinn talk)
If advanced civilization is wiped out by nuclear war, that simulation might be terminated, if it seems sufficiently unlikely to lead to a singularity.
Yep. What I was thinking was: Maybe most simulations of our era are made for the purpose of acausal trade or something like it. And maybe societies that are ravaged by nuclear war make for poor trading partners for some reason. (e.g. maybe they never rebuild, or maybe it takes too long to figure out whether or not they eventually rebuild that it’s not worth the cost of simulating them, or maybe they rebuild but in a way that makes them poor trading partners.) So then the situation would be: Even if most civilizations in our era nuke themselves in a way that doesn’t lead to extinction, the vast majority of people in our era would be in a civilization that didn’t, because they’d be in a simulation of one of the few civilizations that didn’t.
What I’m confused about right now is what the policy implications are of this. As I understand it, the dialectic is something like: A: Nuclear war isn’t worth worrying about because we’ve survived it for 60 years so far, so it must be very unlikely. B: But anthropics! Maybe actually the probability of nuclear war is fairly high. Because of anthropics we’d never know; dead people aren’t observers. A: But nuclear war wouldn’t have killed everyone; if nuclear war is likely, shouldn’t we expect to find ourselves in some post-apocalyptic civilization? Me: But simulations! If post-apocalyptic civilizations are unlikely to be simulated, then it could be that nuclear war is actually pretty likely after all, and we just don’t know because we’re in one of the simulations of the precious few civilizations that avoided nuclear war. Simulations that launch nukes get shut down. Me 2: OK, but… maybe that means that nuclear war is unlikely after all? Or at least, should be treated as unlikely? Me: Why? Me 2: I’m not sure… something something we should ignore hypotheses in which we are simulated because most of our expected impact comes from hypotheses in which we aren’t? Me: That doesn’t seem like it would justify ignoring nuclear war. Look, YOU are the one who has the burden of proof; you need to argue that nuclear war is unlikely on the grounds that it hasn’t happened so far, but I’ve presented a good rebuttal to that argument. Me 2: OK let’s do some math. Two worlds. In World Safe, nuclear war is rare. In World Dangerous, nuclear war is common. In both worlds, most people in our era are simulations and moreover there are no simulations of post-apocalyptic eras. Instead of doing updates, let’s just ask what policy is the best way to hedge our bets between these two worlds… Well, what the simulations do doesn’t matter so much, so we should make a policy that mostly just optimizes for what the non-simulations do. And most of the non-simulations with evidence like ours are in World Safe. So the best policy is to treat nukes as dangerous.
OK, that felt good. I think I tentatively agree with Me 2.
[EDIT: Lol I mean “treat nukes as NOT dangerous/likely” what a typo!]
I feel like this is a very important point that I have never heard made before.
To be clear, after I made it, I thought more about it and I’m not sure it’s correct. I think I’d have to actually do the math, my intuitions aren’t coming in loud and clear here. The reason I’m unsure is that even if for some reason post-apocalyptic worlds rarely get simulated (and thus it’s very unsurprising that we find ourself in a world that didn’t suffer an apocalypse, because we’re probably in a simulation) it may be that we ought to ignore this, since we are trying to act as if we are not simulated anyway, since that’s how we have the most influence or something.
To draw out the argument a little further, the reason that post-apocalyptic worlds don’t get simulated is because most (?) of the simulations of our era are a way to simulate super intelligences in other parts of the multiverse, to talk or trade with.
(As in the basic argument of this Jan Tallinn talk)
If advanced civilization is wiped out by nuclear war, that simulation might be terminated, if it seems sufficiently unlikely to lead to a singularity.
Yep. What I was thinking was: Maybe most simulations of our era are made for the purpose of acausal trade or something like it. And maybe societies that are ravaged by nuclear war make for poor trading partners for some reason. (e.g. maybe they never rebuild, or maybe it takes too long to figure out whether or not they eventually rebuild that it’s not worth the cost of simulating them, or maybe they rebuild but in a way that makes them poor trading partners.) So then the situation would be: Even if most civilizations in our era nuke themselves in a way that doesn’t lead to extinction, the vast majority of people in our era would be in a civilization that didn’t, because they’d be in a simulation of one of the few civilizations that didn’t.
What I’m confused about right now is what the policy implications are of this. As I understand it, the dialectic is something like:
A: Nuclear war isn’t worth worrying about because we’ve survived it for 60 years so far, so it must be very unlikely.
B: But anthropics! Maybe actually the probability of nuclear war is fairly high. Because of anthropics we’d never know; dead people aren’t observers.
A: But nuclear war wouldn’t have killed everyone; if nuclear war is likely, shouldn’t we expect to find ourselves in some post-apocalyptic civilization?
Me: But simulations! If post-apocalyptic civilizations are unlikely to be simulated, then it could be that nuclear war is actually pretty likely after all, and we just don’t know because we’re in one of the simulations of the precious few civilizations that avoided nuclear war. Simulations that launch nukes get shut down.
Me 2: OK, but… maybe that means that nuclear war is unlikely after all? Or at least, should be treated as unlikely?
Me: Why?
Me 2: I’m not sure… something something we should ignore hypotheses in which we are simulated because most of our expected impact comes from hypotheses in which we aren’t?
Me: That doesn’t seem like it would justify ignoring nuclear war. Look, YOU are the one who has the burden of proof; you need to argue that nuclear war is unlikely on the grounds that it hasn’t happened so far, but I’ve presented a good rebuttal to that argument.
Me 2: OK let’s do some math. Two worlds. In World Safe, nuclear war is rare. In World Dangerous, nuclear war is common. In both worlds, most people in our era are simulations and moreover there are no simulations of post-apocalyptic eras. Instead of doing updates, let’s just ask what policy is the best way to hedge our bets between these two worlds… Well, what the simulations do doesn’t matter so much, so we should make a policy that mostly just optimizes for what the non-simulations do. And most of the non-simulations with evidence like ours are in World Safe. So the best policy is to treat nukes as dangerous.
OK, that felt good. I think I tentatively agree with Me 2.
[EDIT: Lol I mean “treat nukes as NOT dangerous/likely” what a typo!]