I see a lot of object-level discussion (I agree with the modal comment) but not much meta.
I am probably the right person to stress that “our current theories of anthropics,” here on LW, are not found in a Bostrom paper.
Our “current theory of anthropics” around these parts (chews on stalk of grass) is simply to start with a third-person model of the world and then condition on your own existence (no need for self-blinding or weird math, just condition on all your information as per normal). The differences in possible world-models and self-information subsumes, explains, and adds shades of grey to Bostrom-paradigm disagreements about “assumption” and “reference class.”
To clarify, do you think I was wrong to say UDT would play the game? I’ve read the two posts you linked. I think I understand Weis, and I think the UDT described there would play. I don’t quite understand yours.
I agree with faul sname, ADifferentAnonymous, shminux, etc. If every single person in the world had to play russian roulette (1 bullet and 5 empty chambers), and the firing pin was broken on exactly one gun in the whole world, everyone except the person with the broken gun would be dead after about 125 trigger pulls.
So if I remember being forced to pull the trigger 1000 times, and I’m still alive, it’s vastly more likely that I’m the one human with the broken gun, or that I’m hallucinating, or something else, rather than me just getting lucky. Note that if you think you might be hallucinating, and you happen to be holding a gun, I recommend putting it down and going for a nap, not pulling the trigger in any way. But for the sake of argument we might suppose the only allowed hypotheses are “working gun” and “broken gun.”
Sure, if there are miraculous survivors, then they will erroneously think that they have the broken gun, in much the same way that if you flipped a coin 1000 times and just so happened to get all heads, you might start to think you had an unfair coin. We should not expect to be able to save this person. They are just doomed.
It’s like poker. I don’t know if you’ve played poker, but you probably know that the basic idea is to make bets that you have the best hand. If you have 4 of a kind, that’s an amazing hand, and you should be happy to make big bets. But it’s still possible for your opponent to have a royal flush. If that’s the case, you’re doomed, and in fact when the opponent has a royal flush, 4 of a kind is almost the worst hand possible! It makes you think you can bet all your money when in fact you’re about to lose it all. It’s precisely the fact that four of a kind is a good hand almost all the time that makes it especially bad that remaining tiny amount of the time.
The person who plays russian roulette and wins 1000 times with a working gun is just that poor sap who has four of a kind into a royal flush.
(P.S.: My post is half explanation of how I would calculate the answer, and half bullet-biting on an unusual anthropic problem. The method has a short summary: just have a probabilistic model of the world and then condition on the existence of yourself (with all your memories, faculties, etc). This gives you the right conditional probability distribution over the world. The complications are because this model has to be a fancy directed graph that has “logical nodes” corresponding to the output of your own decision-making procedure, like in TDT. )
Maybe the disagreement is in how we consider the alternative hypothesis to be? I’m not imagining a broken gun—you could examine your gun and notice it isn’t, or just shoot into the air a few times and see it firing. But even after you eliminate all of those, theres still the hypothesis “I’m special for no discernible reason” (or is there?) that can only be tested anthropically, if at all. And this seems worrying.
Maybe heres a stronger way to formulate it: Consider all the copies of yourself across the multiverse. They will sometimes face situations where they could die. And they will always remember having survived all previous ones. So eventually, all the ones still alive will believe they’re protected by fate or something, and then do something suicidal. Now you can bring the same argument about how there are a few actual immortals, but still… “A rational agent that survives long enough will kill itself unless its literally impossible for it to do so” doesn’t inspire confidence, does it? And it happens even in very “easy” worlds. There is no world where you have a limited chance of dying before you “learn the ropes” and are safe—its impossible to have a chance of eventual death other than 0 or 1, without the laws of nature changing over time.
just have a probabilistic model of the world and then condition on the existence of yourself (with all your memories, faculties, etc).
I interpret that as conditioning on the existence of at least one thing with the “inner” properties of yourself.
The problem, as I understand it, is that there seem to be magical hypothesis you can’t update against from ordinary observation, because by construction the only time they make a difference is in your odds of survival. So you can’t update them from observation, and anthropics can only update in their favour, so eventually you end up believing one and then you die.
I see a lot of object-level discussion (I agree with the modal comment) but not much meta.
I am probably the right person to stress that “our current theories of anthropics,” here on LW, are not found in a Bostrom paper.
Our “current theory of anthropics” around these parts (chews on stalk of grass) is simply to start with a third-person model of the world and then condition on your own existence (no need for self-blinding or weird math, just condition on all your information as per normal). The differences in possible world-models and self-information subsumes, explains, and adds shades of grey to Bostrom-paradigm disagreements about “assumption” and “reference class.”
This is, importantly, the sort of calculation done in UDT/TDT. See e.g. a Wei Dai post, or my later, weirder post.
To clarify, do you think I was wrong to say UDT would play the game? I’ve read the two posts you linked. I think I understand Weis, and I think the UDT described there would play. I don’t quite understand yours.
I agree with faul sname, ADifferentAnonymous, shminux, etc. If every single person in the world had to play russian roulette (1 bullet and 5 empty chambers), and the firing pin was broken on exactly one gun in the whole world, everyone except the person with the broken gun would be dead after about 125 trigger pulls.
So if I remember being forced to pull the trigger 1000 times, and I’m still alive, it’s vastly more likely that I’m the one human with the broken gun, or that I’m hallucinating, or something else, rather than me just getting lucky. Note that if you think you might be hallucinating, and you happen to be holding a gun, I recommend putting it down and going for a nap, not pulling the trigger in any way. But for the sake of argument we might suppose the only allowed hypotheses are “working gun” and “broken gun.”
Sure, if there are miraculous survivors, then they will erroneously think that they have the broken gun, in much the same way that if you flipped a coin 1000 times and just so happened to get all heads, you might start to think you had an unfair coin. We should not expect to be able to save this person. They are just doomed.
It’s like poker. I don’t know if you’ve played poker, but you probably know that the basic idea is to make bets that you have the best hand. If you have 4 of a kind, that’s an amazing hand, and you should be happy to make big bets. But it’s still possible for your opponent to have a royal flush. If that’s the case, you’re doomed, and in fact when the opponent has a royal flush, 4 of a kind is almost the worst hand possible! It makes you think you can bet all your money when in fact you’re about to lose it all. It’s precisely the fact that four of a kind is a good hand almost all the time that makes it especially bad that remaining tiny amount of the time.
The person who plays russian roulette and wins 1000 times with a working gun is just that poor sap who has four of a kind into a royal flush.
(P.S.: My post is half explanation of how I would calculate the answer, and half bullet-biting on an unusual anthropic problem. The method has a short summary: just have a probabilistic model of the world and then condition on the existence of yourself (with all your memories, faculties, etc). This gives you the right conditional probability distribution over the world. The complications are because this model has to be a fancy directed graph that has “logical nodes” corresponding to the output of your own decision-making procedure, like in TDT. )
Maybe the disagreement is in how we consider the alternative hypothesis to be? I’m not imagining a broken gun—you could examine your gun and notice it isn’t, or just shoot into the air a few times and see it firing. But even after you eliminate all of those, theres still the hypothesis “I’m special for no discernible reason” (or is there?) that can only be tested anthropically, if at all. And this seems worrying.
Maybe heres a stronger way to formulate it: Consider all the copies of yourself across the multiverse. They will sometimes face situations where they could die. And they will always remember having survived all previous ones. So eventually, all the ones still alive will believe they’re protected by fate or something, and then do something suicidal. Now you can bring the same argument about how there are a few actual immortals, but still… “A rational agent that survives long enough will kill itself unless its literally impossible for it to do so” doesn’t inspire confidence, does it? And it happens even in very “easy” worlds. There is no world where you have a limited chance of dying before you “learn the ropes” and are safe—its impossible to have a chance of eventual death other than 0 or 1, without the laws of nature changing over time.
I interpret that as conditioning on the existence of at least one thing with the “inner” properties of yourself.
I think in the real world, I am actually accumulating evidence against magic faster than I am trying to commit elaborate suicide.
The problem, as I understand it, is that there seem to be magical hypothesis you can’t update against from ordinary observation, because by construction the only time they make a difference is in your odds of survival. So you can’t update them from observation, and anthropics can only update in their favour, so eventually you end up believing one and then you die.
The amount that I care about this problem is proportional to the chance that I’ll survive to have it.