I really fail to see why you’re all so fascinated by Newcomb-like problems. When you break causality, all logic based on causality doesn’t function any more. If you try to model it mathematically, you will get inconsistent model always.
There’s no need to break causality. You are a being implemented in chaotic wetware. However, there’s no reason to think we couldn’t have rational agents implemented in much more predictable form, as python routines for example, so that any being with superior computation power could simply inspect the source and determine what the output would be.
In such a case, Newcomb-like problems would arise, perfectly lawfully, under normal physics.
In fact, Newcomb-like problems fall naturally out of any ability to simulate and predict the actions of other agents. Omega as described is essentially the limit as predictive power goes to infinity.
This gives me the intuition that trying to decide whether to one-box or two box on newcomb is like trying to decide what 0^0 is; you get your intuition by following a limit process, but that limit process produces different results depending on the path you take.
It would be interesting to look at finitely good predictors. Perhaps we can find something analogous to the result that lim_(x, y -->0) (x^y) is path dependent.
If we define an imperfect predictor as a perfect predictor plus noise, i.e. produces the correct prediction with probability p regardless of the cognition algorithm it’s trying to predict, then Newcomb-like problems are very robust to imperfect prediction: for any p > .5 there is some payoff ratio great enough to preserve the paradox, and the required ratio goes down as the prediction improves. e.g. if 1-boxing gets 100 utilons and 2-boxing gets 1 utilon, then the predictor only needs to be more than 50.5% accurate. So the limit in that direction favors 1-boxing.
What other direction could there be? If the prediction accuracy depends on the algorithm-to-be-predicted (as it would in the real world), then you could try to be an algorithm that is mispredicted in your favor… but a misprediction in your favor can only occur if you actually 2-box, so it only takes a modicum of accuracy before a 1-boxer who tries to be predictable is better off than a 2-boxer who tries to be unpredictable.
I can’t see any other way for the limit to turn out.
If you have two agents trying to precommit not to be blackmailed by each other / precommit not to pay attention to the others precommitment, then any attempt to take a limit of this Newcomblike problem does depend on how you approach the limit. (I don’t know how to solve this problem.)
The value(s) for which the limit is being taken here is unidirectional predictive power, which is loosely a function of the difference in intelligence between the two agents; intuitively, I think a case could be made that (assuming ideal rationality) the total accuracy of mutual behavior prediction between two agents is conserved in some fashion, that doubling the predictive power of one unavoidably would roughly halve the predictive power of the other. Omega represents an entity with a delta-g so large vs. us that predictive power is essentially completely one-sided.
From that basis, allowing the unidirectional predictive power of both agents to go to infinity is probably inherently ill-defined and there’s no reason to expect the problem to have a solution.
there’s no reason to think we couldn’t have rational agents implemented in much more predictable form, as python routines for example, so that any being with superior computation power could simply inspect the source and determine what the output would be.
Such a being would be different from a human in fundamental ways. Imagine knowing with certainty that your actions can be predicted perfectly by the guy next door, even taking into account that you are trying to be hard to predict?
A (quasi)rational agent with access to genuine randomness (such as a human) is a different matter. A superintelligence could almost perfectly predict the probability distribution over my actions, but by quantum entanglement it would not be able to predict my actual actions.
A (quasi)rational agent with access to genuine randomness (such as a human)
Whaddaya mean humans are rational agents with access to genuine randomness? That’s what we’re arguing about in the first place!
A superintelligence could almost perfectly predict the probability distribution over my actions, but by quantum entanglement it would not be able to predict my actual actions.
Perhaps Omega is entangled with your brain such that in all the worlds in which you would choose to one-box, he would predict that you one-box, and all the worlds in which you would choose to two-box, he would predict that you two-box?
Imagine knowing with certainty that your actions can be predicted perfectly by the guy next door, even taking into account that you are trying to be hard to predict?
You wouldn’t know this with certainty* because it wouldn’t be true.
(*unless you were delusional)
The guy next door is on roughly your mental level. Thus, the guy next door can’t predict your actions perfectly, because he can’t run a perfect simulation of your mind that’s faster than you. He doesn’t have the capacity.
And he certainly doesn’t have the capacity to simulate the environment, including other people, while doing so.
A (quasi)rational agent with access to genuine randomness (such as a human) is a different matter.
Humans may or may not generally have access to genuine randomness.
It’s as yet unknown whether we even have run on quantum randomness; and its also unprovable that quantum randomness is actually genuine randomness, and not just based on effects we don’t yet understand, as so many other types of randomness have been.
You wouldn’t know this with certainty* because it wouldn’t be true.
You’re not taking this in the least convenient possible world. Surely it’s not impossible in principle that your neighbor can simulate you and your environment. Perhaps your neighbor is superintelligent?
It’s ALSO not impossible in principle in the real world. A superintelligent entity could, in principle, perfectly predict my actions.
Remember, in the Least Convenient Possible World quantum “randomness” isn’t random.
As such, this ISN’T a fundamental difference between humans and “such beings”.
Which was all I set out to demonstrate.
I was using the “most plausible world” on the basis that it seemed pretty clear that that was the one Roko intended. (Where your neighbour isn’t in fact Yahweh in disguise).
EDIT: Probably should specify worlds for things in this kind of environment. Thanks, the critical environment here is helping me think about how I think/argue.
It’s as yet unknown whether we even have run on quantum randomness; and its also unprovable that quantum randomness is actually genuine randomness, and not just based on effects we don’t yet understand, as so many other types of randomness have been.
If you believe the Many Worlds Interpretation, then quantum randomness just creates copies in a deterministic way.
You cannot do that without breaking Rice’s theorem. If you assume you can find out the answer from someone else’s source code → instant contradiction.
You cannot work around Rice’s theorem or around causality by specifying 50.5% accuracy independently of modeled system, any accuracy higher than 50%+epsilon is equivalent to indefinitely good accuracy by repeatedly predicting (standard cryptographic result), and 50%+epsilon doesn’t cause the paradox.
Give me one serious math model of Newcomb-like problems where the paradox emerges while preserving causality. Here are some examples. Then you model it, you either get trivial solution to one-box, or causality break, or omega loses.
You decide first what you would do in every situation, omega decides second, and now you only implement your initial decision table and are not allowed to switch. Game theory says you should implement one-boxing.
You decide first what you would do in every situation, omega decides second, and now you are allowed to switch. Game theory says you should precommit to one-box, then implement two-boxing, omega loses.
You decide first what you would do in every situation, omega decides second, and now you are allowed to switch. If omega always decides correctly, then he bases his decision on your switch, which either turns it into model #1 (you cannot really switch, precommitment is binding), or breaks causality.
Rice’s theorem says you can’t predict every possible algorithm in general. Plenty of particular algorithms can be predictable. If you’re running on a classical computer and Omega has a copy of you, you are perfectly predictable.
And all of your choices are just as real as they ever were, see the OB sequence on free will (I think someone referred to it already).
And the argument that omega just needs predictive power of 50.5% to cause the paradox only works if it works against ANY arbitrary algorithm. Having that power against any arbitrary algorithm breaks Rice’s Theorem, having that power (or even 100%) against just limited subset of algorithms doesn’t cause the paradox.
If you take strict decision tree precommitment interpretation, then you fix causality. You decide first, omega decides second, game theory says one-box, problem solved.
Decision tree precommitment is never a problem in game theory, as precommitment of the entire tree commutes with decisions by other agents:
A decides what f(X), f(Y) to do if B does X or Y. B does X. A does f(X)
B does X. A decides what f(X), f(Y) to do if B does X or Y. A does f(X)
are identical, as B cannot decide based on f. So the changing your mind problem never occurs.
With omega:
A decides what f(X), f(Y) to do if B does X or Y. B does X. A does f(X) - B can answer depending on f
B does X. A decides what f(X), f(Y) to do if B does X or Y. A does f(X) - somehow not allowed any more
I don’t think the paradox exist in any plausible mathematization of the problem. It looks to me like another of those philosophical problems that exist because of sloppiness of natural language and very little more, I’m just surprised that OB/LW crowd cares about this one and not about others. OK, I admit I really enjoyed it the first time I saw it but just as something fun, nothing more than that.
They don’t require breaking causality. The argument works if Omega is barely predicting you above chance. I’m sure there are plenty of normal people who can do that just by talking to you.
There are also more important reasons. Take the doomsday argument. You can use the fact that you’re alive now to predict that we’ll die out “soon”. Suppose you had a choice between saving a life in a third-world country that likely wouldn’t amount to anything, or donating to SIAI to help in the distant future. You know it’s very unlikely for there to be a distant future. It’s like Omega did his coin toss, and if it comes up tails, we die out early and he asks you to waste the money by donating to SIAI. If it comes up heads, you’re in the future, and it’s better if you would have donated.
That’s not some thing that might happen. That’s a decision you have to make before you pick a charity to donate to. Lives are riding on this. That’s if the coin lands on tails. If it lands on heads, there is more life riding on it than has so far existed in the known universe. Please choose carefully.
The argument works if Omega is barely predicting you above chance.
Arguments like these remind me of students’ mistakes from Algorithms and Data Structures 101 - statements like that are very intuitive, absolutely wrong, and once you figure out why this reasoning doesn’t work it’s easy to forget that most people didn’t go through this ever.
What is required is Omega predicting better than chance in the worst case. Predicting correctly with ridiculously tiny chance of error against “average” person is worthless.
To avoid Omega and causality silliness, and just demonstrate this intuition—let’s take a slightly modified version of Boolean satisfiability—but instead of one formula we have three formulas of the same length. If all three are identical, return true or false depending on its satisfiability, if they’re different return true if number of one bits in problem is odd (or some other trivial property).
It is obviously NP-complete, as any satisfiability problem reduces to it by concatenating it three times. If we use exponential brute force to solve the hard case, average running time is O(n) for scanning the string plus O(2^(n/3)) for brute forcing but only 2^-(2n/3) of the time, that is O(1). So we can solve NP-complete problems in average linear time.
What happened? We were led astray by intuition, and assumed that problems that are difficult in worst case cannot be trivial on average. But this equal weighting is an artifact—if you tried reducing any other NP problem into this, you’d be getting very difficult ones nearly all the time, as if by magic.
Back to Omega—even if Omega predicts normal people very well, as long as there are any thinking being who is cannot predict—Omega must break causality. And such being are not just hypothetical—people who decide based on a coin toss are exactly like that. Silly rules about disallowing chance merely make counterexamples more complicated, Omega and Newcomb are still as much based on sloppy thinking as ever.
I don’t know any reason why a coin toss would be the best choice in Newcomb’s paradox. If you decide based on reason, and don’t decide to flip a coin, and Omega knows you well, he can predict your action above chance. The paradox stands.
Omega cannot know coin flip results without violating causality. So he either puts that million in the box or not. As a result, no matter which way he decides, Omega has 50% chance of violating own rules, which was supposedly impossible, breaking the problem.
What I mean is, if you change the scenario so he only has to predict above chance if you don’t flip a coin, and he isn’t always getting it right anyway, the same basic principle applies, but it doesn’t violate causality.
In Bayesian interpretation P() would be Omega’s subjective probability. In frequentist interpretation, the question doesn’t make any sense as you make a single boxing decision, not large number of tiny boxing decisions. Either way P() is very ill-defined.
No more so than other probabilities. Probabilities about future decisions of other actors aren’t disprivileged, that would be free will confusion. And are you seriously claiming that the probabilities of a coin flip don’t make sense in a frequentist interpretation? That was the context. In the general case it would be the long term relative frequency of possible versions of you similar enough to you to be indistinguishable for Omega deciding that way or something like that, if you insisted on using frequentist statistics for some reason.
You misunderstand frequentist interpretation—sample size is 1 - you either decide yes or decide no. To generalize from a single decider needs prior reference class (“toin cosses”), getting us into Bayesian subjective interpretations. Frequentists don’t have any concept of “probability of hypothesis” at all, only “probability of data given hypothesis” and the only way to connect them is using priors. “Frequency among possible worlds” is also a Bayesian thing that weirds frequentists out.
Anyway, if Omega has amazing prediction powers, and P() can be deterministically known by looking into the box this is far more valuable than mere $1,000,000! Let’s say I make my decision by randomly generating some string and checking if it’s a valid proof of Riemann hypothesis—if P() is non-zero, I made myself $1,000,000 anyway.
I understand that there’s an obvious technical problem if Omega rounds the number to whole dollars, but that’s just minor detail.
And actually, it is a lot worse in popular problem formulation of “if your decision relies on randomness, there will be no million” that tries to work around coin tossing. In such case a person randomly trying to prove false statement gets a million (as no proof could work, so his decision was reliable), and a person randomly trying to prove true statement gets $0 (as there’s non-zero chance of him randomly generating correct proof).
Another fun idea would be measuring both position and velocity of an electron—tossing a coin to decide either way, measuring one and getting the other from Omega.
The issue was whether the formulation makes sense, not whether it makes frequentialists freak out (and it’s not substantially different than e. g. drawing from an urn for the first time). In either case P() was the probablitity of an event, not a hypothesis.
In these sorts of problems you are supposed to assume that the dollar amounts match your actual utilities (as you observe your exploit doesn’t work anyway for tests with a probability of <0.5*10^-9 if rounding to cents, and you could just assume that you already have gained all knowledge you could gain through such test, or that Omega possesses exactly the same knowledge as you except for human psychology, or whatever).
I really fail to see why you’re all so fascinated by Newcomb-like problems.
Agreed. This problem seems uninteresting to me too. Though more realistic newcomb-like problems are interesting; for there are parts of life where newcombian reasoning works for real.
On second thoughts, since many clever philosophers spend careers on these problems, I may be missing something.
The obvious complaint about “would you choose X or Y given that Omega already knows your actions” is that it is logically inconsistent; if Omega already knows your actions, the word “choose” is nonsense. Strictly speaking, “choose” is nonsense anyway; it takes the naive free will point of view in its everyday usage.
In order to untangle this, a sophisticated understanding of what we mean by “choose” is needed. I may post on this. My intuition is that if we stick to a rigorous meaning of “choose”, the question will have a well-defined answer that no-one will dispute, however what this answer is will depend on the definition of “choose” that you, um, choose, so to speak…
This problem seems uninteresting to me too. Though more realistic newcomb-like problems are interesting; for there are parts of life where newcombian reasoning works for real.
I find the problem interesting, so I’ll try to explain why I find it interesting.
So there are these blogs called Overcoming Bias and Less Wrong, and the people posting on it seem like very smart people, and they say very reasonable things. They offer to teach how to become rational, in the sense of “winning more often”. I want to win more often too, so I read the blogs.
Now a lot of what these people are saying sounds very reasonable, but it’s also clear that the people saying these things are much smarter than me; so much so that although their conclusions sound very reasonable, I can’t always follow all the arguments or steps used to reach those conclusions. As part of my rationalist training, I try to notice when I can follow the steps to a conclusion, and when I can’t, and remember which conclusions I believe in because I fully understand it, and which conclusions I am “tentatively believing in” because someone smart said it, and I’m just taking their word for it for now.
So now Vladimir Nesov presents this puzzle, and I realize that I must not have understood one of the conclusions (or I did understand them, and the smart people were mistaken), because it sounds like if I were to follow the advice of this blog, I’d be doing something really stupid (depending on how you answered VN’s problem, the stupid thing is either “wasting $100” or “wasting $4950″).
So how do I reconcile this with everything I’ve learned on this blog?
Think of most of the blog as a textbook, with VN’s post being an “exercise to the reader” or a “homework problem”.
I think I’m not confused about free will, and that the links I gave should help to resolve most of the confusion. Maybe you should write a blog post/LW article where you formulate the nature of your confusion (if you still have it after reading the relevant material), I’ll respond to that.
Not really—all that is neccessary is that Omega is a sufficiently accurate predictor that the payoff matrix, taking this accuracy into question, still amounts to a win for the given choice. There is no need to be a perfect predictor. And if an imperfect, 99.999% predictor violates free will, then it’s clearly a lost cause anyway (I can predict with similar precision many behaviours about people based on no more evidence than their behaviour and speech, never mind godlike brain introspection) Do you have no “choice” in deciding to come to work tomorrow, if I predict based on your record that you’re 99.99% reliable? Where is the cut-off that free will gets lost?
Do you have no “choice” in deciding to come to work tomorrow, if I predict based on your record that you’re 99.99% reliable?
Humans are subtle beasts. If you tell me that you have predicted that I will go to work based upon my 99.99% attendance record, the probability that I will go to work drops dramatically upon me receiving that information, because there is a good chance that I’ll not go just to be awkward. This option of “taking your prediction into account, I’ll do the opposite to be awkward” is why it feels like you have free will.
Chances are I can predict such a response too, and so won’t tell you of my prediction (or tell you in such a way that you will be more likely to attend: eg. “I’ve a $50 bet you’ll attend tomorrow. Be there and I’ll split it 50:50”). It doesn’t change the fact that in this particular instance I can fortell the future with a high degree of accuracy. Why then would it violate free will if Omega could predict your accuracy in this different situation (one where he’s also able to predict the effects of him telling you) to a similar precision?
Why then would it violate free will if Omega could predict your accuracy in this different situation (one where he’s also able to predict the effects of him telling you) to a similar precision?
Because that’s pretty much our intuitive definition of free will; that it is not possible for someone to predict your actions, announce it publicly, and still be correct. If you disagree, we are disagreeing about the intuitive definition of “free will” that most people carry around in their heads. At least admit that most people would be unsurprised if a person predicted that they would (e.g.) brush their teeth in the morning (without telling them in advance that it had predicted that), versus predicting that they would knock a vase over, and then as a result of that prediction, the vase actually getting knocked over.
Then take my bet situation. I announce your attendance, and cut you in with a $25 stake in attendance. I don’t think it would be unusual to find someone who would indeed appear 99.99% of the time—does that mean that person has no free will?
People are highly, though not perfectly, predictable under a large number of situations. Revealing knowledge about the prediction complicates things by adding feedback to the system, but there are lots of cases where it still doesn’t change matters much (or even increases predictability). There are obviously some situations where this doesn’t happen, but for Newcombe’s paradox, all that is needed is a predictor for the particular situation described, not any general situation. (In fact Newcombe’s paradox is equally broken by a similar revelation of knowledge. If Omega were to reveal its prediction before the boxes are chosen, a person determined to do the opposite of that prediction opens it up to a simple Epimenides paradox.)
I really fail to see why you’re all so fascinated by Newcomb-like problems. When you break causality, all logic based on causality doesn’t function any more. If you try to model it mathematically, you will get inconsistent model always.
There’s no need to break causality. You are a being implemented in chaotic wetware. However, there’s no reason to think we couldn’t have rational agents implemented in much more predictable form, as python routines for example, so that any being with superior computation power could simply inspect the source and determine what the output would be.
In such a case, Newcomb-like problems would arise, perfectly lawfully, under normal physics.
In fact, Newcomb-like problems fall naturally out of any ability to simulate and predict the actions of other agents. Omega as described is essentially the limit as predictive power goes to infinity.
This gives me the intuition that trying to decide whether to one-box or two box on newcomb is like trying to decide what 0^0 is; you get your intuition by following a limit process, but that limit process produces different results depending on the path you take.
It would be interesting to look at finitely good predictors. Perhaps we can find something analogous to the result that lim_(x, y -->0) (x^y) is path dependent.
If we define an imperfect predictor as a perfect predictor plus noise, i.e. produces the correct prediction with probability p regardless of the cognition algorithm it’s trying to predict, then Newcomb-like problems are very robust to imperfect prediction: for any p > .5 there is some payoff ratio great enough to preserve the paradox, and the required ratio goes down as the prediction improves. e.g. if 1-boxing gets 100 utilons and 2-boxing gets 1 utilon, then the predictor only needs to be more than 50.5% accurate. So the limit in that direction favors 1-boxing.
What other direction could there be? If the prediction accuracy depends on the algorithm-to-be-predicted (as it would in the real world), then you could try to be an algorithm that is mispredicted in your favor… but a misprediction in your favor can only occur if you actually 2-box, so it only takes a modicum of accuracy before a 1-boxer who tries to be predictable is better off than a 2-boxer who tries to be unpredictable.
I can’t see any other way for the limit to turn out.
If you have two agents trying to precommit not to be blackmailed by each other / precommit not to pay attention to the others precommitment, then any attempt to take a limit of this Newcomblike problem does depend on how you approach the limit. (I don’t know how to solve this problem.)
The value(s) for which the limit is being taken here is unidirectional predictive power, which is loosely a function of the difference in intelligence between the two agents; intuitively, I think a case could be made that (assuming ideal rationality) the total accuracy of mutual behavior prediction between two agents is conserved in some fashion, that doubling the predictive power of one unavoidably would roughly halve the predictive power of the other. Omega represents an entity with a delta-g so large vs. us that predictive power is essentially completely one-sided.
From that basis, allowing the unidirectional predictive power of both agents to go to infinity is probably inherently ill-defined and there’s no reason to expect the problem to have a solution.
Such a being would be different from a human in fundamental ways. Imagine knowing with certainty that your actions can be predicted perfectly by the guy next door, even taking into account that you are trying to be hard to predict?
A (quasi)rational agent with access to genuine randomness (such as a human) is a different matter. A superintelligence could almost perfectly predict the probability distribution over my actions, but by quantum entanglement it would not be able to predict my actual actions.
Whaddaya mean humans are rational agents with access to genuine randomness? That’s what we’re arguing about in the first place!
Perhaps Omega is entangled with your brain such that in all the worlds in which you would choose to one-box, he would predict that you one-box, and all the worlds in which you would choose to two-box, he would predict that you two-box?
In the original formulation, if Omega expects you to flip a coin, he leaves box B empty.
You wouldn’t know this with certainty* because it wouldn’t be true.
(*unless you were delusional)
The guy next door is on roughly your mental level. Thus, the guy next door can’t predict your actions perfectly, because he can’t run a perfect simulation of your mind that’s faster than you. He doesn’t have the capacity.
And he certainly doesn’t have the capacity to simulate the environment, including other people, while doing so.
Humans may or may not generally have access to genuine randomness.
It’s as yet unknown whether we even have run on quantum randomness; and its also unprovable that quantum randomness is actually genuine randomness, and not just based on effects we don’t yet understand, as so many other types of randomness have been.
You’re not taking this in the least convenient possible world. Surely it’s not impossible in principle that your neighbor can simulate you and your environment. Perhaps your neighbor is superintelligent?
It’s ALSO not impossible in principle in the real world. A superintelligent entity could, in principle, perfectly predict my actions. Remember, in the Least Convenient Possible World quantum “randomness” isn’t random.
As such, this ISN’T a fundamental difference between humans and “such beings”. Which was all I set out to demonstrate.
I was using the “most plausible world” on the basis that it seemed pretty clear that that was the one Roko intended. (Where your neighbour isn’t in fact Yahweh in disguise). EDIT: Probably should specify worlds for things in this kind of environment. Thanks, the critical environment here is helping me think about how I think/argue.
If you believe the Many Worlds Interpretation, then quantum randomness just creates copies in a deterministic way.
You cannot do that without breaking Rice’s theorem. If you assume you can find out the answer from someone else’s source code → instant contradiction.
You cannot work around Rice’s theorem or around causality by specifying 50.5% accuracy independently of modeled system, any accuracy higher than 50%+epsilon is equivalent to indefinitely good accuracy by repeatedly predicting (standard cryptographic result), and 50%+epsilon doesn’t cause the paradox.
Give me one serious math model of Newcomb-like problems where the paradox emerges while preserving causality. Here are some examples. Then you model it, you either get trivial solution to one-box, or causality break, or omega loses.
You decide first what you would do in every situation, omega decides second, and now you only implement your initial decision table and are not allowed to switch. Game theory says you should implement one-boxing.
You decide first what you would do in every situation, omega decides second, and now you are allowed to switch. Game theory says you should precommit to one-box, then implement two-boxing, omega loses.
You decide first what you would do in every situation, omega decides second, and now you are allowed to switch. If omega always decides correctly, then he bases his decision on your switch, which either turns it into model #1 (you cannot really switch, precommitment is binding), or breaks causality.
Rice’s theorem says you can’t predict every possible algorithm in general. Plenty of particular algorithms can be predictable. If you’re running on a classical computer and Omega has a copy of you, you are perfectly predictable.
And all of your choices are just as real as they ever were, see the OB sequence on free will (I think someone referred to it already).
And the argument that omega just needs predictive power of 50.5% to cause the paradox only works if it works against ANY arbitrary algorithm. Having that power against any arbitrary algorithm breaks Rice’s Theorem, having that power (or even 100%) against just limited subset of algorithms doesn’t cause the paradox.
If you take strict decision tree precommitment interpretation, then you fix causality. You decide first, omega decides second, game theory says one-box, problem solved.
Decision tree precommitment is never a problem in game theory, as precommitment of the entire tree commutes with decisions by other agents:
A decides what f(X), f(Y) to do if B does X or Y. B does X. A does f(X)
B does X. A decides what f(X), f(Y) to do if B does X or Y. A does f(X)
are identical, as B cannot decide based on f. So the changing your mind problem never occurs.
With omega:
A decides what f(X), f(Y) to do if B does X or Y. B does X. A does f(X) - B can answer depending on f
B does X. A decides what f(X), f(Y) to do if B does X or Y. A does f(X) - somehow not allowed any more
I don’t think the paradox exist in any plausible mathematization of the problem. It looks to me like another of those philosophical problems that exist because of sloppiness of natural language and very little more, I’m just surprised that OB/LW crowd cares about this one and not about others. OK, I admit I really enjoyed it the first time I saw it but just as something fun, nothing more than that.
I don’t know why nobody mentioned this at the time, but that’s hardly an unpopular view around here (as I’m sure you’ve noticed by now).
The interesting thing about Newcomb had nothing to do with thinking it was a genuine paradox—just counterintuitive for some.
They don’t require breaking causality. The argument works if Omega is barely predicting you above chance. I’m sure there are plenty of normal people who can do that just by talking to you.
There are also more important reasons. Take the doomsday argument. You can use the fact that you’re alive now to predict that we’ll die out “soon”. Suppose you had a choice between saving a life in a third-world country that likely wouldn’t amount to anything, or donating to SIAI to help in the distant future. You know it’s very unlikely for there to be a distant future. It’s like Omega did his coin toss, and if it comes up tails, we die out early and he asks you to waste the money by donating to SIAI. If it comes up heads, you’re in the future, and it’s better if you would have donated.
That’s not some thing that might happen. That’s a decision you have to make before you pick a charity to donate to. Lives are riding on this. That’s if the coin lands on tails. If it lands on heads, there is more life riding on it than has so far existed in the known universe. Please choose carefully.
Arguments like these remind me of students’ mistakes from Algorithms and Data Structures 101 - statements like that are very intuitive, absolutely wrong, and once you figure out why this reasoning doesn’t work it’s easy to forget that most people didn’t go through this ever.
What is required is Omega predicting better than chance in the worst case. Predicting correctly with ridiculously tiny chance of error against “average” person is worthless.
To avoid Omega and causality silliness, and just demonstrate this intuition—let’s take a slightly modified version of Boolean satisfiability—but instead of one formula we have three formulas of the same length. If all three are identical, return true or false depending on its satisfiability, if they’re different return true if number of one bits in problem is odd (or some other trivial property).
It is obviously NP-complete, as any satisfiability problem reduces to it by concatenating it three times. If we use exponential brute force to solve the hard case, average running time is O(n) for scanning the string plus O(2^(n/3)) for brute forcing but only 2^-(2n/3) of the time, that is O(1). So we can solve NP-complete problems in average linear time.
What happened? We were led astray by intuition, and assumed that problems that are difficult in worst case cannot be trivial on average. But this equal weighting is an artifact—if you tried reducing any other NP problem into this, you’d be getting very difficult ones nearly all the time, as if by magic.
Back to Omega—even if Omega predicts normal people very well, as long as there are any thinking being who is cannot predict—Omega must break causality. And such being are not just hypothetical—people who decide based on a coin toss are exactly like that. Silly rules about disallowing chance merely make counterexamples more complicated, Omega and Newcomb are still as much based on sloppy thinking as ever.
I don’t know any reason why a coin toss would be the best choice in Newcomb’s paradox. If you decide based on reason, and don’t decide to flip a coin, and Omega knows you well, he can predict your action above chance. The paradox stands.
Omega cannot know coin flip results without violating causality. So he either puts that million in the box or not. As a result, no matter which way he decides, Omega has 50% chance of violating own rules, which was supposedly impossible, breaking the problem.
What I mean is, if you change the scenario so he only has to predict above chance if you don’t flip a coin, and he isn’t always getting it right anyway, the same basic principle applies, but it doesn’t violate causality.
The obvious extensions of the problem to cases with failable Omega are:
P( $1,000,000) = P(onebox)
Reward = $1,000,000 * P(onebox)
In Bayesian interpretation P() would be Omega’s subjective probability. In frequentist interpretation, the question doesn’t make any sense as you make a single boxing decision, not large number of tiny boxing decisions. Either way P() is very ill-defined.
No more so than other probabilities. Probabilities about future decisions of other actors aren’t disprivileged, that would be free will confusion. And are you seriously claiming that the probabilities of a coin flip don’t make sense in a frequentist interpretation? That was the context. In the general case it would be the long term relative frequency of possible versions of you similar enough to you to be indistinguishable for Omega deciding that way or something like that, if you insisted on using frequentist statistics for some reason.
(this comment assumes “Reward = $1,000,000 * P(onebox)”)
You misunderstand frequentist interpretation—sample size is 1 - you either decide yes or decide no. To generalize from a single decider needs prior reference class (“toin cosses”), getting us into Bayesian subjective interpretations. Frequentists don’t have any concept of “probability of hypothesis” at all, only “probability of data given hypothesis” and the only way to connect them is using priors. “Frequency among possible worlds” is also a Bayesian thing that weirds frequentists out.
Anyway, if Omega has amazing prediction powers, and P() can be deterministically known by looking into the box this is far more valuable than mere $1,000,000! Let’s say I make my decision by randomly generating some string and checking if it’s a valid proof of Riemann hypothesis—if P() is non-zero, I made myself $1,000,000 anyway.
I understand that there’s an obvious technical problem if Omega rounds the number to whole dollars, but that’s just minor detail.
And actually, it is a lot worse in popular problem formulation of “if your decision relies on randomness, there will be no million” that tries to work around coin tossing. In such case a person randomly trying to prove false statement gets a million (as no proof could work, so his decision was reliable), and a person randomly trying to prove true statement gets $0 (as there’s non-zero chance of him randomly generating correct proof).
Another fun idea would be measuring both position and velocity of an electron—tossing a coin to decide either way, measuring one and getting the other from Omega.
Possibilities are just endless.
The issue was whether the formulation makes sense, not whether it makes frequentialists freak out (and it’s not substantially different than e. g. drawing from an urn for the first time). In either case P() was the probablitity of an event, not a hypothesis.
In these sorts of problems you are supposed to assume that the dollar amounts match your actual utilities (as you observe your exploit doesn’t work anyway for tests with a probability of <0.5*10^-9 if rounding to cents, and you could just assume that you already have gained all knowledge you could gain through such test, or that Omega possesses exactly the same knowledge as you except for human psychology, or whatever).
Agreed. This problem seems uninteresting to me too. Though more realistic newcomb-like problems are interesting; for there are parts of life where newcombian reasoning works for real.
On second thoughts, since many clever philosophers spend careers on these problems, I may be missing something.
The obvious complaint about “would you choose X or Y given that Omega already knows your actions” is that it is logically inconsistent; if Omega already knows your actions, the word “choose” is nonsense. Strictly speaking, “choose” is nonsense anyway; it takes the naive free will point of view in its everyday usage.
In order to untangle this, a sophisticated understanding of what we mean by “choose” is needed. I may post on this. My intuition is that if we stick to a rigorous meaning of “choose”, the question will have a well-defined answer that no-one will dispute, however what this answer is will depend on the definition of “choose” that you, um, choose, so to speak…
I find the problem interesting, so I’ll try to explain why I find it interesting.
So there are these blogs called Overcoming Bias and Less Wrong, and the people posting on it seem like very smart people, and they say very reasonable things. They offer to teach how to become rational, in the sense of “winning more often”. I want to win more often too, so I read the blogs.
Now a lot of what these people are saying sounds very reasonable, but it’s also clear that the people saying these things are much smarter than me; so much so that although their conclusions sound very reasonable, I can’t always follow all the arguments or steps used to reach those conclusions. As part of my rationalist training, I try to notice when I can follow the steps to a conclusion, and when I can’t, and remember which conclusions I believe in because I fully understand it, and which conclusions I am “tentatively believing in” because someone smart said it, and I’m just taking their word for it for now.
So now Vladimir Nesov presents this puzzle, and I realize that I must not have understood one of the conclusions (or I did understand them, and the smart people were mistaken), because it sounds like if I were to follow the advice of this blog, I’d be doing something really stupid (depending on how you answered VN’s problem, the stupid thing is either “wasting $100” or “wasting $4950″).
So how do I reconcile this with everything I’ve learned on this blog?
Think of most of the blog as a textbook, with VN’s post being an “exercise to the reader” or a “homework problem”.
The primary reason for resolving Newcomb-like problems is to explore the fundamental limitations of decision theories.
It sounds like you are still confused about free will. See Righting a Wrong Question, Possibility and Could-ness, and Daniel Dennett’s lecture here.
yes, I am confused about free will, but I think that this confusion is legitimate given our current lack of knowledge about how the human mind works.
I hope I’m not making obvious errors about free will. But if I am, then I’d like to know...
I think I’m not confused about free will, and that the links I gave should help to resolve most of the confusion. Maybe you should write a blog post/LW article where you formulate the nature of your confusion (if you still have it after reading the relevant material), I’ll respond to that.
Not really—all that is neccessary is that Omega is a sufficiently accurate predictor that the payoff matrix, taking this accuracy into question, still amounts to a win for the given choice. There is no need to be a perfect predictor. And if an imperfect, 99.999% predictor violates free will, then it’s clearly a lost cause anyway (I can predict with similar precision many behaviours about people based on no more evidence than their behaviour and speech, never mind godlike brain introspection) Do you have no “choice” in deciding to come to work tomorrow, if I predict based on your record that you’re 99.99% reliable? Where is the cut-off that free will gets lost?
Humans are subtle beasts. If you tell me that you have predicted that I will go to work based upon my 99.99% attendance record, the probability that I will go to work drops dramatically upon me receiving that information, because there is a good chance that I’ll not go just to be awkward. This option of “taking your prediction into account, I’ll do the opposite to be awkward” is why it feels like you have free will.
Chances are I can predict such a response too, and so won’t tell you of my prediction (or tell you in such a way that you will be more likely to attend: eg. “I’ve a $50 bet you’ll attend tomorrow. Be there and I’ll split it 50:50”). It doesn’t change the fact that in this particular instance I can fortell the future with a high degree of accuracy. Why then would it violate free will if Omega could predict your accuracy in this different situation (one where he’s also able to predict the effects of him telling you) to a similar precision?
Because that’s pretty much our intuitive definition of free will; that it is not possible for someone to predict your actions, announce it publicly, and still be correct. If you disagree, we are disagreeing about the intuitive definition of “free will” that most people carry around in their heads. At least admit that most people would be unsurprised if a person predicted that they would (e.g.) brush their teeth in the morning (without telling them in advance that it had predicted that), versus predicting that they would knock a vase over, and then as a result of that prediction, the vase actually getting knocked over.
Then take my bet situation. I announce your attendance, and cut you in with a $25 stake in attendance. I don’t think it would be unusual to find someone who would indeed appear 99.99% of the time—does that mean that person has no free will?
People are highly, though not perfectly, predictable under a large number of situations. Revealing knowledge about the prediction complicates things by adding feedback to the system, but there are lots of cases where it still doesn’t change matters much (or even increases predictability). There are obviously some situations where this doesn’t happen, but for Newcombe’s paradox, all that is needed is a predictor for the particular situation described, not any general situation. (In fact Newcombe’s paradox is equally broken by a similar revelation of knowledge. If Omega were to reveal its prediction before the boxes are chosen, a person determined to do the opposite of that prediction opens it up to a simple Epimenides paradox.)
On second thoughts, since many clever philosophers spend careers on these problems, I may be missing something.
Nah, they just need something to talk about.