“I just flipped a fair coin. I decided, before I flipped the coin, that if it came up heads, I would ask you for $1000. And if it came up tails, I would give you $1,000,000 if and only if I predicted that you would give me $1000 if the coin had come up heads. The coin came up heads—can I have $1000?”
Obviously, the only reflectively consistent answer in this case is “Yes—here’s the $1000”, because if you’re an agent who expects to encounter many problems like this in the future, you will self-modify to be the sort of agent who answers “Yes” to this sort of question—just like with Newcomb’s Problem or Parfit’s Hitchhiker.
I don’t understand why “Yes” is the right answer. It seems to me that an agent that self-modified to answer “Yes” to this sort of question in the future but said “No” this time would generate more utility than an agent that already implemented the policy of saying yes.
If I was going to insert an agent into the universe at the moment the question was posed after the coin flip had occurred, I would place one that answered “No” this time, but answered “Yes” in the future. (Assuming I have no information other than the information provided in the problem description.)
When you get to the future, would you regret having to answer “Yes”, given that if you weren’t so rational you could just answer “No”? If so, you should answer “No” every time. What is the difference between this time and the future? From your present position, you see all future possibilities, and so you make tradeoffs between them, acting on expected utility. But at present, you could similarly take notice of the alternative possibilities that are “sideways” from where you are, things that could happen to you but didn’t, and similarly act on expected utility. There doesn’t seem to be any fundamental reason for discarding (ceasing to care about) possibilities on the basis of not happening to be located within them, it’s just practical to do so since you normally can’t do anything about them.
I don’t understand why “Yes” is the right answer. It seems to me that an agent that self-modified to answer “Yes” to this sort of question in the future but said “No” this time would generate more utility than an agent that already implemented the policy of saying yes.
If I was going to insert an agent into the universe at the moment the question was posed after the coin flip had occurred, I would place one that answered “No” this time, but answered “Yes” in the future. (Assuming I have no information other than the information provided in the problem description.)
When you get to the future, would you regret having to answer “Yes”, given that if you weren’t so rational you could just answer “No”? If so, you should answer “No” every time. What is the difference between this time and the future? From your present position, you see all future possibilities, and so you make tradeoffs between them, acting on expected utility. But at present, you could similarly take notice of the alternative possibilities that are “sideways” from where you are, things that could happen to you but didn’t, and similarly act on expected utility. There doesn’t seem to be any fundamental reason for discarding (ceasing to care about) possibilities on the basis of not happening to be located within them, it’s just practical to do so since you normally can’t do anything about them.
(See Counterfactual Mugging and UDT for more discussion.)
This strategy is not reflectively consistent. From the new TDT PDF:
If an agent implemented your strategy, they would change decision strategies every time they come across a predictor that flipped heads.