The main point of FDT is that it gives the optimal expected utility on average for agents using it. It does not guarantee optimal expected utility for every instance of an agent using it.
Suppose you have a population of two billion agents, each going through this scenario every day. Upon seeing a note predicting right, one billion would pick left and one billion would pick right. We can assume that they all pick left if they see a note predicting left or no note at all.
Every year, the Right agents essentially always see a note predicting right, and pay more than $30000 each. The Left agents essentially always see a note predicting left (or no note) and pay $0 each.
The average rate of deaths is comparable: one death per few trillion years in each group, which is to say, essentially never. They all know that it could happen, of course.
Which group is better off?
Edit: I misread Predictor’s accuracy. It does not say that it is in all scenarios 1 − 10^-24, just that in some unknown sample of scenarios, it was 1 − 10^-24. This changes the odds so much that FDT does not recommend taking the left box.
Edit: I misread Predictor’s accuracy. It does not say that it is in all scenarios 1 − 10^-24, just that in some unknown sample of scenarios, it was 1 − 10^-24. This changes the odds so much that FDT does not recommend taking the left box.
Two questions, if I may:
Why do you read it this way? The problem simply states the failure rate is 1 in a trillion trillion.
If we go with your interpretation, why exactly does that change things? It seems to me that the sample size would have to be extemely huge in order to determine a failure rate that low.
It depends upon what the meaning of the word “is” is:
The failure rate has been tested over an immense number of prediction, and evaluated as 10^-24 (to one significant figure). That is the currently accepted estimate for the predictor’s error rate for scenarios randomly selected from the sample.
The failure rate is theoretically 10^-24, over some assumed distribution of agent types. Your decision model may or may not appear anywhere in this distribution.
The failure rate is bounded above by 10^-24 for every possible scenario.
A self-harming agent in this scenario cannot be consistently predicted by Predictor at all (success rate 0%), so we know that (3) is definitely false.
(1) and (2) aren’t strong enough, because it gives little information about Predictor’s error rate concerning your scenario and your decision model.
We have essentially zero information about Predictor’s true error bounds regarding agents that sometimes carry out self-harming actions. In order to recommend taking the left box, an FDT agent is one that sometimes carries out self-harming actions, though this requires that the upper bound on Predictor’s failure of subjunctive dependency is less than the ratio of the utilities of: paying $100, and burning to death all intelligent life in the universe.
We do not have anywhere near enough information to justify that tight a bound. So FDT can’t recommend such an action. Maybe someone else can write a scenario that is in similar spirit, but isn’t so flawed.
Another way of phrasing it: you don’t get the $100 marginal payoff if you’re not prepared to knowingly go to your death in the incredibly unlikely event of a particular type of misprediction.
That’s the sense in which I meant “you got screwed”. You entered the scenario knowing that it was incredibly unlikely that you would die regardless of what you decide, but were prepared to accept that incredibly microscopic chance of death in exchange for keeping your $100. The odds just went against you.
Edit: If Predictor’s actual bound on error rate was 10^-24, this would be valid. However, Predictor’s bound on error rate cannot be 10^-24 in all scenarios, so this is all irrelevant. What a waste of time.
The main point of FDT is that it gives the optimal expected utilityon averagefor agents using it. It does not guarantee optimal expected utility forevery instanceof an agent using it.Suppose you have a population of two billion agents, each going through this scenario every day. Upon seeing a note predicting right, one billion would pick left and one billion would pick right. We can assume that they all pick left if they see a note predicting left or no note at all.Every year, the Right agents essentially always see a note predicting right, and pay more than $30000 each. The Left agents essentially always see a note predicting left (or no note) and pay $0 each.The average rate of deaths is comparable: one death per few trillion years in each group, which is to say, essentially never. They all know that itcouldhappen, of course.Which group is better off?Edit: I misread Predictor’s accuracy. It does not say that it is in all scenarios 1 − 10^-24, just that in some unknown sample of scenarios, it was 1 − 10^-24. This changes the odds so much that FDT does not recommend taking the left box.
Obviously, the group that’s better off is the third group: the one that picks Left if there’s no bomb in there, Right otherwise.
… I mean, seriously, what the heck? The scenario specifies that the boxes are open! You can see what’s in there! How is this even a question?
(Bonus question: what will the predictor say about the behavior of this third group? What choice will she predict a member of this group will make?)
Two questions, if I may:
Why do you read it this way? The problem simply states the failure rate is 1 in a trillion trillion.
If we go with your interpretation, why exactly does that change things? It seems to me that the sample size would have to be extemely huge in order to determine a failure rate that low.
It depends upon what the meaning of the word “is” is:
The failure rate has been tested over an immense number of prediction, and evaluated as 10^-24 (to one significant figure). That is the currently accepted estimate for the predictor’s error rate for scenarios randomly selected from the sample.
The failure rate is theoretically 10^-24, over some assumed distribution of agent types. Your decision model may or may not appear anywhere in this distribution.
The failure rate is bounded above by 10^-24 for every possible scenario.
A self-harming agent in this scenario cannot be consistently predicted by Predictor at all (success rate 0%), so we know that (3) is definitely false.
(1) and (2) aren’t strong enough, because it gives little information about Predictor’s error rate concerning your scenario and your decision model.
We have essentially zero information about Predictor’s true error bounds regarding agents that sometimes carry out self-harming actions. In order to recommend taking the left box, an FDT agent is one that sometimes carries out self-harming actions, though this requires that the upper bound on Predictor’s failure of subjunctive dependency is less than the ratio of the utilities of: paying $100, and burning to death all intelligent life in the universe.
We do not have anywhere near enough information to justify that tight a bound. So FDT can’t recommend such an action. Maybe someone else can write a scenario that is in similar spirit, but isn’t so flawed.
Thanks, I appreciate this. Your answer clarifies a lot, and I will think about it more.
Another way of phrasing it: you don’t get the $100 marginal payoff if you’renot preparedto knowingly go to your death in the incredibly unlikely event of a particular type of misprediction.That’s the sense in which I meant “you got screwed”. You entered the scenario knowing that it was incredibly unlikely that you would die regardless of what you decide, but wereprepared to acceptthat incredibly microscopic chance of death in exchange for keeping your $100. The odds just went against you.Edit: If Predictor’s actual bound on error rate was 10^-24, this would be valid. However, Predictor’s bound on error rate cannot be 10^-24 in all scenarios, so this is all irrelevant. What a waste of time.