Eliezer’s argument, if I understand it, is that any decision-making algorithm that results in two-boxing is by definition irrational due to giving a predictably bad outcome.
So he’s assuming the conclusion that you get a bad outcome? Golly.
grobstein
Karma: 119
This premise is not accepted by the 1-box contingent. Occasionally they claim there’s a reason.
Please … Newcomb is a toy non-mathematizable problem and not a valid argument for anything at all.
Why?
Actually the problem is an ambiguity in “right”—you can take the “right” course of action (instrumental rationality, or ethics), or you can have “right” belief (epistemic rationality).
Here’s a functional difference: Omega says that Box B is empty if you try to win what’s inside it.
Your argument is equivalent to, “But what if your utility function rates keeping promises higher than a million orgasms, what then?”
The hypo is meant to be a very simple model, because simple models are useful. It includes two goods: getting home, and having $100. Any other speculative values that a real person might or might not have are distractions.
Right. The question of course is, “better” for what purpose? Which model is better depends on what you’re trying to figure out.
I do think these problems are mostly useful for purposes of understanding and (moreso) defining rationality (“rationality”), which is perhaps a somewhat dubious use. But look how much time we’re spending on it.
I very much recommend Reasons and Persons, by the way. A friend stole my copy and I miss it all the time.
What is it, pray tell, that Omega cannot do?
Can he not scan your brain and determine what strategy you are following? That would be odd, because this is no stronger than the original Newcomb problem and does not seem to contain any logical impossibilities.
Can he not compute the strategy, S, with the property “that at each moment, acting as S tells you to act—given (1) your beliefs about the universe at that point and (2) your intention of following S at all times—maximizes your net utility [over all time]?” That would be very odd, since you seem to believe a regular person can compute S. If you can do it, why not Omega? (NB, no, it doesn’t help to define an approximation of S and use that. If it’s rational, Omega will punish you for it. If it’s not, why are you doing it?)
Can he not compare your strategy to S, given that he knows the value of each? That seems odd, because a pushdown automaton could make the comparison. Do you require Omega to be weaker than a pushdown automaton?
No?
Then is it possible, maybe, that the problem is in the definition of S?
Yes, this seems unimpeachable. The missing piece is, rational at what margin? Once you are home, it is not rational at the margin to pay the $100 you promised.
It’s a test case for rationality as pure self-interest (really it’s like an altruistic version of the game of Chicken).
Suppose I’m purely selfish and stranded on a road at night. A motorist pulls over and offers to take me home for $100, which is a good deal for me. I only have money at home. I will be able to get home then IFF I can promise to pay $100 when I get home.
But when I get home, the marginal benefit to paying $100 is zero (under assumption of pure selfishness). Therefore if I behave rationally at the margin when I get home, I cannot keep my promise.
I am better off overall if I can commit in advance to keeping my promise. In other words, I am better off overall if I have a disposition which sometimes causes me to behave irrationally at the margin. Under the self-interest notion of rationality, then, it is rational, at the margin of choosing your disposition, to choose a disposition which is not rational under the self-interest notion of rationality. (This is what Parfit describes as an “indirectly self-defeating” result; note that being indirectly self-defeating is not a knockdown argument against a position.)
Yes, you are changing the hypo. Your Omega dummy says that it is the same game as Newcomb’s problem, but it’s not. As VN notes, it may be equivalent to the version of Newcomb’s problem that assumes time travel, but this is not the classical (or an interesting) statement of the problem.
No. The point is that you actually want to survive more than you want to win, so if you are rational about Chicken you will sometimes lose (consult your model for details). Given your preferences, there will always be some distance \epsilon before the cliff where it is rational for you to give up.
Therefore, under these assumptions, the strategy “win or die trying” seemingly requires you to be irrational. However, if you can credibly commit to this strategy—be the kind of person who will win or die trying—you will beat a rational player every time.
This is a case where it is rational to have an irrational disposition, a disposition other than doing what is rational at every margin.
Why don’t you accept his distinction between acting rationally at a given moment and having the disposition which it is rational to have, integrated over all time?
EDIT: er, Parfit’s, that is.
Obviously we can construct an agent who does this. I just don’t see a reasonably parsimonious model that does it without including a preference for getting AIDS, or something similarly crazy. Perhaps I’m just stuck.
(likewise the fairness language of the parent post)
It’s impossible to add substance to “non-pathological universe.” I suspect circularity: a non-pathological universe is one that rewards rationality; rationality is the disposition that lets you win in a nonpathological universe.
You need to attempt to define terms to avoid these traps.
This is a classic point and clearer than the related argument I’m making above. In addition to being part of the accumulated game theory learning, it’s one of the types of arguments that shows up frequently in Derek Parfit’s discussion of what-is-rationality, in Ch. 1 of Reasons and Persons.
I feel like there are difficulties here that EY is not attempting to tackle.
Rationality is made of win.
Duhhh!
(Cf.)