What if the person being asked for the $100 is a simulation of you which Omega is using to check whether you’ll pay if he asked you? You won’t know whether you’re the simulation or not.
To predict, Omega doesn’t need to simulate. You can predict that water will boil when put on fire without simulating the movement of 10^23 molecules.
Omega even can’t use simulation to arrive at his prediction in this scenario. If Omega demands money from simulated agents who then agree to pay, the simulation violates the formulation of the problem, according to which Omega should reward those agents.
If the problem is reformulated as “Omega demands payment only if the agent would counterfactually disagree to pay, OR in a simulation”, then we have a completely different problem. For example, if the agent is sufficiently confident about his own decision algorithm, then after Omega’s demand he could assign high probability to being in a simulation. The analysis would be more complicated there.
In short, I am only saying that
Omega is trustworthy.
Omega can predict the agents behaviour with certainty.
Omega tells that it demands money only from agents whom it predicted to reject the demand.
You can predict that water will boil when put on fire without simulating the movement of 10^23 molecules.
True but irrelevant. In order to make an accurate prediction, Omega needs, at the very least, to simulate my decision-making faculty in all significant aspects. If my decision-making process decides to recall some particular memory, then Omega needs to simulate that memory in all significant aspects. If my decision-making process decides to wander around the room conducting physics experiments, just to be a jackass, and to peg my decision to the results of those experiments—well, then Omega will need to convincingly simulate the results of those experiments. The anticipated experience will be identical for my actual decision-making process as for my simulated decision-making process.
Mind you, based on what I know of the brain, I think you’d actually need to run a pretty convincing, if somewhat coarse-grained, simulation of a good chunk of my light cone in order to predict my decision with any kind of certainty, but I’m being charitable here.
And yes, this seems to render the original formulation of the problem paradoxical. I’m trying to think of ways to suitably reformulate it without altering the decision theoretics, but I’m not sure it’s possible.
True but irrelevant. In order to make an accurate prediction, Omega needs, at the very least, to simulate my decision-making faculty in all significant aspects. If my decision-making process decides to recall some particular memory, then Omega needs to simulate that memory in all significant aspects. If my decision-making process decides to wander around the room conducting physics experiments, just to be a jackass, and to peg my decision to the results of those experiments—well, then Omega will need to convincingly simulate the results of those experiments.
I’m not convinced that all that actually follows from the premises. One of the features of Newcomblike problems is that they tend to appear intuitively obvious to the people exposed to them, which suggests rather strongly to me that the intuitive answer is linked to hidden variables in personality or experience, and in most cases isn’t sensitively dependent on initial conditions.
People don’t always choose the intuitive answer, of course, but augmenting that with information about the decision-theoretic literature you’ve been exposed to, any contrarian tendencies you might have, etc. seems like it might be sufficient to achieve fine-grained predictive power without actually running a full simulation of you. The better the predictive power, of course, the more powerful the model of your decision-making process has to be, but Omega doesn’t actually have to have perfect predictive power for Newcomblike conditions to hold. It doesn’t even have to have particularly good predictive power, given the size of the payoff.
Er, I think we’re talking about two different formulations of the problem (both of which are floating around on this page, so this isn’t too surprising). In the original post, the constraint is given by P(o=award)=P(a=pay), rather than P(o=award)=qP(a=pay)+(1-q)P(a=refuse), which implies that Omega’s prediction is nearly infallible, as it usually is in problems starring Omega: any deviation from P(o=award)=0 or 1 will be due to “truly random” influences on my decision (e.g. quantum coin tosses). Also, I think the question is not “what are your intuitions?” but “what is the optimal decision for a rationalist in these circumstances?”
You seem to be suggesting that most of what determines my decision to pay or refuse could be boiled down to a few factors. I think the evidence weighs heavily against this: effect sizes in psychological studies tend to be very weak. Evidence also suggests that these kinds of cognitive processes are indeed sensitively dependent on initial conditions. Differences in the way questions are phrased, and what you’ve had on your mind lately, can have a significant impact, just to name a couple of examples.
That is, at that point it sounds like it’s no longer “Omega will ask me to pay him $100 if he predicts I wouldn’t pay him if he asked,” it’s “Omega will ask me to pay him $100 if he predicts I wouldn’t pay him if he asked OR if I’m a simulation.”
Not that any of this is necessary. Willingness to pay Omega depends on having arbitrarily high confidence in his predictions; it’s not clear that I could ever arrive at such a high level of confidence, but it doesn’t matter.
We’re just asking, if it’s true, what decision on my part maximizes expected results for entities for which I wish to maximize expected results? Perhaps I could never actually be expected to make that decision because I could never be expected to have sufficient confidence that it’s true. That changes nothing.
Also worth noting that if I pay him $100 in this scenario I ought to update my confidence level sharply downward. That is, if I’ve previously seen N predictions and Omega has been successful in each of them, I have now seen (N+1) predictions and he’s been successful in N of them.
Of course, by then I’ve already paid; there’s no longer a choice to make.
(Presumably I should believe, prior to agreeing, that my choosing to pay him will not actually result in my paying him, or something like that… I ought not expect to pay him, given that he’s offered me $100, regardless of what choice I make. In which case I might as well choose to pay him. This is absurd, of course, but the whole situation is absurd.)
ETA—I am apparently confused on more fundamental levels than I had previously understood, not least of which is what is being presumed about Omega in these cases. Apparently I am not presumed to be as confident of Omega’s predictions as I’d thought, which makes the rest of this comment fairly irrelevant. Oops.
What if the person being asked for the $100 is a simulation of you which Omega is using to check whether you’ll pay if he asked you? You won’t know whether you’re the simulation or not.
To predict, Omega doesn’t need to simulate. You can predict that water will boil when put on fire without simulating the movement of 10^23 molecules.
Omega even can’t use simulation to arrive at his prediction in this scenario. If Omega demands money from simulated agents who then agree to pay, the simulation violates the formulation of the problem, according to which Omega should reward those agents.
If the problem is reformulated as “Omega demands payment only if the agent would counterfactually disagree to pay, OR in a simulation”, then we have a completely different problem. For example, if the agent is sufficiently confident about his own decision algorithm, then after Omega’s demand he could assign high probability to being in a simulation. The analysis would be more complicated there.
In short, I am only saying that
Omega is trustworthy.
Omega can predict the agents behaviour with certainty.
Omega tells that it demands money only from agents whom it predicted to reject the demand.
Omega demands the money.
The agent pays.
are together incompatible statements.
True but irrelevant. In order to make an accurate prediction, Omega needs, at the very least, to simulate my decision-making faculty in all significant aspects. If my decision-making process decides to recall some particular memory, then Omega needs to simulate that memory in all significant aspects. If my decision-making process decides to wander around the room conducting physics experiments, just to be a jackass, and to peg my decision to the results of those experiments—well, then Omega will need to convincingly simulate the results of those experiments. The anticipated experience will be identical for my actual decision-making process as for my simulated decision-making process.
Mind you, based on what I know of the brain, I think you’d actually need to run a pretty convincing, if somewhat coarse-grained, simulation of a good chunk of my light cone in order to predict my decision with any kind of certainty, but I’m being charitable here.
And yes, this seems to render the original formulation of the problem paradoxical. I’m trying to think of ways to suitably reformulate it without altering the decision theoretics, but I’m not sure it’s possible.
I’m not convinced that all that actually follows from the premises. One of the features of Newcomblike problems is that they tend to appear intuitively obvious to the people exposed to them, which suggests rather strongly to me that the intuitive answer is linked to hidden variables in personality or experience, and in most cases isn’t sensitively dependent on initial conditions.
People don’t always choose the intuitive answer, of course, but augmenting that with information about the decision-theoretic literature you’ve been exposed to, any contrarian tendencies you might have, etc. seems like it might be sufficient to achieve fine-grained predictive power without actually running a full simulation of you. The better the predictive power, of course, the more powerful the model of your decision-making process has to be, but Omega doesn’t actually have to have perfect predictive power for Newcomblike conditions to hold. It doesn’t even have to have particularly good predictive power, given the size of the payoff.
Er, I think we’re talking about two different formulations of the problem (both of which are floating around on this page, so this isn’t too surprising). In the original post, the constraint is given by P(o=award)=P(a=pay), rather than P(o=award)=qP(a=pay)+(1-q)P(a=refuse), which implies that Omega’s prediction is nearly infallible, as it usually is in problems starring Omega: any deviation from P(o=award)=0 or 1 will be due to “truly random” influences on my decision (e.g. quantum coin tosses). Also, I think the question is not “what are your intuitions?” but “what is the optimal decision for a rationalist in these circumstances?”
You seem to be suggesting that most of what determines my decision to pay or refuse could be boiled down to a few factors. I think the evidence weighs heavily against this: effect sizes in psychological studies tend to be very weak. Evidence also suggests that these kinds of cognitive processes are indeed sensitively dependent on initial conditions. Differences in the way questions are phrased, and what you’ve had on your mind lately, can have a significant impact, just to name a couple of examples.
Doesn’t that contradict the original assertion?
That is, at that point it sounds like it’s no longer “Omega will ask me to pay him $100 if he predicts I wouldn’t pay him if he asked,” it’s “Omega will ask me to pay him $100 if he predicts I wouldn’t pay him if he asked OR if I’m a simulation.”
Not that any of this is necessary. Willingness to pay Omega depends on having arbitrarily high confidence in his predictions; it’s not clear that I could ever arrive at such a high level of confidence, but it doesn’t matter.
We’re just asking, if it’s true, what decision on my part maximizes expected results for entities for which I wish to maximize expected results? Perhaps I could never actually be expected to make that decision because I could never be expected to have sufficient confidence that it’s true. That changes nothing.
Also worth noting that if I pay him $100 in this scenario I ought to update my confidence level sharply downward. That is, if I’ve previously seen N predictions and Omega has been successful in each of them, I have now seen (N+1) predictions and he’s been successful in N of them.
Of course, by then I’ve already paid; there’s no longer a choice to make.
(Presumably I should believe, prior to agreeing, that my choosing to pay him will not actually result in my paying him, or something like that… I ought not expect to pay him, given that he’s offered me $100, regardless of what choice I make. In which case I might as well choose to pay him. This is absurd, of course, but the whole situation is absurd.)
ETA—I am apparently confused on more fundamental levels than I had previously understood, not least of which is what is being presumed about Omega in these cases. Apparently I am not presumed to be as confident of Omega’s predictions as I’d thought, which makes the rest of this comment fairly irrelevant. Oops.
No. Paying is the winning strategy in the version where the predictor is correct only with probability, say, 0.8, too. ie.
(blink)
You’re right; I’m wrong. Clearly I haven’t actually been thinking carefully about the problem.
Thanks.