Curses on this problem; I spent the whole day worrying about it, and am now so much of a wreck that the following may or may not make sense. For better or worse, I came to a similar conclusion of Psy-Kosh: that this could work in less anthropic problems. Here’s the equivalent I was using:
Imagine Omega has a coin biased so that it comes up the same way nine out of ten times. You know this, but you don’t know which way it’s biased. Omega allows you to flip the coin once, and asks for your probability that it’s biased in favor of heads. The coin comes up heads. You give your probability as 9⁄10.
Now Omega takes 20 people and puts them in the same situation as in the original problem. It lets each of them flip their coins. Then it goes to each of the people who got tails, and offers $1 to charity for each coin that came up tails, but threatens to steal $3 from charity for each coin that came up heads.
This nonanthropic problem works the same way as the original anthropic problem. If the coin is really biased heads, 18 people will get heads and 2 people will get tails. In this case,the correct subjective probability to assign is definitely 9⁄10 in favor of whatever result you got; after all, this is the correct probability when you’re the only person in the experiment, and just knowing that 19 other people are also participating in the experiment shouldn’t change matters.
I don’t have a formal answer for why this happens, but I can think of one more example that might throw a little light on it. In another thread, someone mentioned that lottery winners have excellent evidence that they are brains-in-a-vat and that the rest of the world is an illusion being put on by the Dark Lord of the Matrix for their entertainment. After all, if this was true, it wouldn’t be too unlikely for them to win the lottery, so for a sufficiently large lottery, the chance of winning it this way exceeds the chance of winning it through luck.
Suppose Bob has won the lottery and so believes himself to be a brain in a vat. And suppose that the evidence for the simulation argument is poor enough that there is no other good reason to believe yourself to be a brain in a vat. Omega goes up to Bob and asks him to take a bet on whether he is a brain in a vat. Bob says he is, he loses, and Omega laughs at him. What did he do wrong? Nothing. Omega was just being mean by specifically asking the one person whom ve knew would get the answer wrong.
Omega’s little prank would still work if ve announced ver intention to perform it beforehand. Ve would say “When one of you wins the lottery, I will be asking this person to take a bet whether they are a brain in a vat or not!” Everyone would say “That lottery winner shouldn’t accept Omega’s bet. We know we’re not brains in vats.” Then someone wins the lottery, Omega asks if they’re a brain in a vat, and they say yes, and Omega laughs at them (note that this also works if we consider a coin with a bias such that it lands the same way 999999 out of a million times, let a million people flip it once, and ask people what they think the coin’s bias is, asking the people who get the counter-to-expectations result more often than chance.)
Omega’s being equally mean in the original problem. There’s a 50% chance ve will go and ask the two out of twenty people who are specifically most likely to be wrong and can’t do anything about it. The best course I can think of would be for everyone to swear an oath not to take the offer before they got assigned into rooms.
Then someone wins the lottery, Omega asks if they’re a brain in a vat, and they say yes, and Omega laughs at them
By assumption, if the person is right to believe they’re in a sim, then most of the lottery winners are in sims, so while Omega laughs at them in our world, they win the bet with Omega in most of their worlds.
This is a feature of the original problem, isn’t it?
Let’s say there are 1000 brains in vats, each in their own little world, and a “real” world of a billion people. The chance of a vat-brain winning the lottery is 1, and the chance of a real person winning the lottery is 1 in a million. There are 1000 real lottery winners and 1000 vat lottery winners, so if you win the lottery your chance of being in a vat is 50-50. However, if you look at any particular world, the chances of this week’s single lottery winner being a brain in a vat is 1000/1001.
Assume the original problem is run multiple times in multiple worlds, and that the value of pi somehow differs in those worlds (probably you used pi precisely so people couldn’t do this, but bear with me). Of all the people who wake up in green rooms, 18⁄20 of them will be right to take your bet. However, in each particular world, the chances of the green room people being right to take the bet is 1⁄2.
In this situation there is no paradox. Most of the people in the green rooms come out happy that they took the bet. It’s only when you limit it to one universe that it becomes a problem. The same is true of the lottery example. When restricted to a single (real, non-vat) universe, it becomes more troublesome.
Now Omega takes 20 people and puts them in the same situation as in the original problem. It lets each of them flip their coins. Then it goes to each of the people who got tails, and offers $1 to charity for each coin that came up tails, but threatens to steal $3 from charity for each coin that came up heads.
It’s worth noting that if everyone got to make this choice separately—Omega doing it once for each person who responds—then it would indeed be wise for everyone to take the bet! This is evidence in favor of either Bostrom’s division-of-responsibility principle, or byrnema’s pointer-based viewpoint, if indeed those two views are nonequivalent.
Bostrom’s calculation is correct, but I believe it is an example of multiplying by the right coefficients for the wrong reasons.
I did exactly the same thing—multiplied by the right coefficients for the wrong reasons—in my deleted comment. I realized that the justification of these coefficients required a quite different problem (in my case, I modeled that all the green roomers decided to evenly divide the spoils of the whole group) and the only reason it worked was because multiplying the first term by 1⁄18 and the next term by 1⁄2 meant you were effectively canceling away that the factors the represented your initial 90% posterior, and thus ultimately just applying the 50⁄50 probability of the non-anthropic solution.
Anthropic calculation:
18/20(12)+2/20(-52) = 5.6
Bostrom-modified calculation for responsibility per person:
[18/20(12)/18+2/20(-52)/2] / 2 = −1
Non-anthropic calculation for EV per person:
[1/2(12)+1/2(-52)] /20 = −1
My pointer-based viewpoint, in contrast, is not a calculation but a rationale for why you must use the 50⁄50 probability rather than the 90⁄10 one. The argument is that each green roomer cannot use the information that they were in a green room because this information was preselected (a biased sample). With effectively no information about what color room they’re in, each green roomer must resort to the non-anthropic calculation that the probability of flipping heads is 50%.
I can very much relate to Eliezer’s original gut reaction: I agree that Nick’s calculation is very ad hoc and hardly justifiable.
However, I also think that, although you are right about the pointer bias, your explanation is still incomplete.
I think Psi-kosh made an important step with his reformulation. Especially eliminating the copy procedure for the agents was essential. If you follow through the math from the point of view of one of the agents, the nature of the problem becomes clear:
Trying to write down the payoff matrix from the viewpoint of one of the agents, it becomes clear that you can’t fill out any of the reward entries, since the outcome never depends on that agent’s decision alone. If he got a green marble, it still depends on other agents decision and if he drew a red one, it will depend only on other agent’s decision.
This makes it completely clear that the only solution is for the agents is to agree on a predetermined protocol and therefore the second calculation of the OP is the only correct one so far.
However, this protocol does not imply anything about P(head|being in green room). It is simply irrelevant for the expected value of any of the agreed upon protocol. One could create a protocol that depends on P(head|being in a green room) for some of the agents, but you would have to analyze the expected value of the protocol from a global point of view, not just from the point of view of the agent, for you can’t complete the decision matrix if the outcome depends on other agent’s decisions as well.
Of course a predetermined protocol does not mean that the agents must explicitly agree on a narrow protocol before the action. If we assume that the agents get all the information once they find themselves in the room, they could still create a mental model of the whole global situation and base their decision on the second calculation of the OP.
I agree with you that the reason why you can’t use the 90⁄10 prior is because the decision never depends on a person in a red room.
In Eliezer’s description of the problem above, he tells each green roomer that he asks all the green roomers if they want him to go ahead with a money distribution scheme, and they must be unanimous or there is a penalty.
I think this is a nice pedogogical component that helps a person understand the dilemma, but I would like to emphasize here (even if you’re aware of it) that it is completely superfluous to the mechanics of the problem. It doesn’t make any difference if Eliezer bases his action on the answer of one green roomer or all of them.
For one thing, all green roomer answers will be unanimous because they all have the same information and are asked the same complicated question.
And, more to the point, even if just one green roomer is asked, the dilemma still exists that he can’t use his prior that heads was probably flipped.
[EDIT:] Although I would be a bit more general: regardless of red rooms: if you have several actors, even if they necessarily make the same decision they have to analyze the global picture. The only situation when the agent should be allowed to make the simplified subjective Bayesian decision table analysis if he is the only actor (no copies, etc. It is easy to construct simple decision problems without “red rooms”: Where each of the actors have some control over the outcome and none of them can make the analysis for itself only but have to buid a model of the whole situation to make the globally optimal decision.)
However, I did not imply in any way that the penalty matters. (At least, as long as the agents are sane and don’t start to flip non-logical coins) The global analysis of the payoff may clearly disregard the penalty case if it’s impossible for that specific protocol. The only requirement is that the expected value calculation must be made protocol by protocol basis.
My intuition says that this is qualitatively different. If the agent knows that only one green roomer will be asked the question, then upon waking up in a green room the agent thinks “with 90% probability, there are 18 of me in green rooms and 2 of me in red rooms.” But then, if the agent is asked whether to take the bet, this new information (“I am the unique one being asked”) changes the probability back to 50-50.
Curses on this problem; I spent the whole day worrying about it, and am now so much of a wreck that the following may or may not make sense. For better or worse, I came to a similar conclusion of Psy-Kosh: that this could work in less anthropic problems. Here’s the equivalent I was using:
Imagine Omega has a coin biased so that it comes up the same way nine out of ten times. You know this, but you don’t know which way it’s biased. Omega allows you to flip the coin once, and asks for your probability that it’s biased in favor of heads. The coin comes up heads. You give your probability as 9⁄10.
Now Omega takes 20 people and puts them in the same situation as in the original problem. It lets each of them flip their coins. Then it goes to each of the people who got tails, and offers $1 to charity for each coin that came up tails, but threatens to steal $3 from charity for each coin that came up heads.
This nonanthropic problem works the same way as the original anthropic problem. If the coin is really biased heads, 18 people will get heads and 2 people will get tails. In this case,the correct subjective probability to assign is definitely 9⁄10 in favor of whatever result you got; after all, this is the correct probability when you’re the only person in the experiment, and just knowing that 19 other people are also participating in the experiment shouldn’t change matters.
I don’t have a formal answer for why this happens, but I can think of one more example that might throw a little light on it. In another thread, someone mentioned that lottery winners have excellent evidence that they are brains-in-a-vat and that the rest of the world is an illusion being put on by the Dark Lord of the Matrix for their entertainment. After all, if this was true, it wouldn’t be too unlikely for them to win the lottery, so for a sufficiently large lottery, the chance of winning it this way exceeds the chance of winning it through luck.
Suppose Bob has won the lottery and so believes himself to be a brain in a vat. And suppose that the evidence for the simulation argument is poor enough that there is no other good reason to believe yourself to be a brain in a vat. Omega goes up to Bob and asks him to take a bet on whether he is a brain in a vat. Bob says he is, he loses, and Omega laughs at him. What did he do wrong? Nothing. Omega was just being mean by specifically asking the one person whom ve knew would get the answer wrong.
Omega’s little prank would still work if ve announced ver intention to perform it beforehand. Ve would say “When one of you wins the lottery, I will be asking this person to take a bet whether they are a brain in a vat or not!” Everyone would say “That lottery winner shouldn’t accept Omega’s bet. We know we’re not brains in vats.” Then someone wins the lottery, Omega asks if they’re a brain in a vat, and they say yes, and Omega laughs at them (note that this also works if we consider a coin with a bias such that it lands the same way 999999 out of a million times, let a million people flip it once, and ask people what they think the coin’s bias is, asking the people who get the counter-to-expectations result more often than chance.)
Omega’s being equally mean in the original problem. There’s a 50% chance ve will go and ask the two out of twenty people who are specifically most likely to be wrong and can’t do anything about it. The best course I can think of would be for everyone to swear an oath not to take the offer before they got assigned into rooms.
By assumption, if the person is right to believe they’re in a sim, then most of the lottery winners are in sims, so while Omega laughs at them in our world, they win the bet with Omega in most of their worlds.
should have been your clue to check further.
This is a feature of the original problem, isn’t it?
Let’s say there are 1000 brains in vats, each in their own little world, and a “real” world of a billion people. The chance of a vat-brain winning the lottery is 1, and the chance of a real person winning the lottery is 1 in a million. There are 1000 real lottery winners and 1000 vat lottery winners, so if you win the lottery your chance of being in a vat is 50-50. However, if you look at any particular world, the chances of this week’s single lottery winner being a brain in a vat is 1000/1001.
Assume the original problem is run multiple times in multiple worlds, and that the value of pi somehow differs in those worlds (probably you used pi precisely so people couldn’t do this, but bear with me). Of all the people who wake up in green rooms, 18⁄20 of them will be right to take your bet. However, in each particular world, the chances of the green room people being right to take the bet is 1⁄2.
In this situation there is no paradox. Most of the people in the green rooms come out happy that they took the bet. It’s only when you limit it to one universe that it becomes a problem. The same is true of the lottery example. When restricted to a single (real, non-vat) universe, it becomes more troublesome.
It’s worth noting that if everyone got to make this choice separately—Omega doing it once for each person who responds—then it would indeed be wise for everyone to take the bet! This is evidence in favor of either Bostrom’s division-of-responsibility principle, or byrnema’s pointer-based viewpoint, if indeed those two views are nonequivalent.
EDIT: Never mind
Bostrom’s calculation is correct, but I believe it is an example of multiplying by the right coefficients for the wrong reasons.
I did exactly the same thing—multiplied by the right coefficients for the wrong reasons—in my deleted comment. I realized that the justification of these coefficients required a quite different problem (in my case, I modeled that all the green roomers decided to evenly divide the spoils of the whole group) and the only reason it worked was because multiplying the first term by 1⁄18 and the next term by 1⁄2 meant you were effectively canceling away that the factors the represented your initial 90% posterior, and thus ultimately just applying the 50⁄50 probability of the non-anthropic solution.
Anthropic calculation:
18/20(12)+2/20(-52) = 5.6
Bostrom-modified calculation for responsibility per person:
[18/20(12)/18+2/20(-52)/2] / 2 = −1
Non-anthropic calculation for EV per person:
[1/2(12)+1/2(-52)] /20 = −1
My pointer-based viewpoint, in contrast, is not a calculation but a rationale for why you must use the 50⁄50 probability rather than the 90⁄10 one. The argument is that each green roomer cannot use the information that they were in a green room because this information was preselected (a biased sample). With effectively no information about what color room they’re in, each green roomer must resort to the non-anthropic calculation that the probability of flipping heads is 50%.
I can very much relate to Eliezer’s original gut reaction: I agree that Nick’s calculation is very ad hoc and hardly justifiable.
However, I also think that, although you are right about the pointer bias, your explanation is still incomplete.
I think Psi-kosh made an important step with his reformulation. Especially eliminating the copy procedure for the agents was essential. If you follow through the math from the point of view of one of the agents, the nature of the problem becomes clear:
Trying to write down the payoff matrix from the viewpoint of one of the agents, it becomes clear that you can’t fill out any of the reward entries, since the outcome never depends on that agent’s decision alone. If he got a green marble, it still depends on other agents decision and if he drew a red one, it will depend only on other agent’s decision.
This makes it completely clear that the only solution is for the agents is to agree on a predetermined protocol and therefore the second calculation of the OP is the only correct one so far.
However, this protocol does not imply anything about P(head|being in green room). It is simply irrelevant for the expected value of any of the agreed upon protocol. One could create a protocol that depends on P(head|being in a green room) for some of the agents, but you would have to analyze the expected value of the protocol from a global point of view, not just from the point of view of the agent, for you can’t complete the decision matrix if the outcome depends on other agent’s decisions as well.
Of course a predetermined protocol does not mean that the agents must explicitly agree on a narrow protocol before the action. If we assume that the agents get all the information once they find themselves in the room, they could still create a mental model of the whole global situation and base their decision on the second calculation of the OP.
I agree with you that the reason why you can’t use the 90⁄10 prior is because the decision never depends on a person in a red room.
In Eliezer’s description of the problem above, he tells each green roomer that he asks all the green roomers if they want him to go ahead with a money distribution scheme, and they must be unanimous or there is a penalty.
I think this is a nice pedogogical component that helps a person understand the dilemma, but I would like to emphasize here (even if you’re aware of it) that it is completely superfluous to the mechanics of the problem. It doesn’t make any difference if Eliezer bases his action on the answer of one green roomer or all of them.
For one thing, all green roomer answers will be unanimous because they all have the same information and are asked the same complicated question.
And, more to the point, even if just one green roomer is asked, the dilemma still exists that he can’t use his prior that heads was probably flipped.
Agreed 100%.
[EDIT:] Although I would be a bit more general: regardless of red rooms: if you have several actors, even if they necessarily make the same decision they have to analyze the global picture. The only situation when the agent should be allowed to make the simplified subjective Bayesian decision table analysis if he is the only actor (no copies, etc. It is easy to construct simple decision problems without “red rooms”: Where each of the actors have some control over the outcome and none of them can make the analysis for itself only but have to buid a model of the whole situation to make the globally optimal decision.)
However, I did not imply in any way that the penalty matters. (At least, as long as the agents are sane and don’t start to flip non-logical coins) The global analysis of the payoff may clearly disregard the penalty case if it’s impossible for that specific protocol. The only requirement is that the expected value calculation must be made protocol by protocol basis.
My intuition says that this is qualitatively different. If the agent knows that only one green roomer will be asked the question, then upon waking up in a green room the agent thinks “with 90% probability, there are 18 of me in green rooms and 2 of me in red rooms.” But then, if the agent is asked whether to take the bet, this new information (“I am the unique one being asked”) changes the probability back to 50-50.