Newcomb’s Problem is silly. It’s only controversial because it’s dressed up in wooey vagueness. In the end it’s just a simple probability question and I’m surprised it’s even taken seriously here. To see why, keep your eyes on the bolded text:
Omega has been correct on each of 100 observed occasions so far—everyone [on each of 100 observed occasions] who took both boxes has found box B empty and received only a thousand dollars; everyone who took only box B has found B containing a million dollars.
What can we anticipate from the bolded part? The only actionable belief we have at this point is that 100 out of 100 times, one-boxing made the one-boxer rich. The details that the boxes were placed by Omega and that Omega is a “superintelligence” add nothing. They merely confuse the matter by slipping in the vague connotation that Omega could be omniscient or something.
In fact, this Omega character is superfluous; the belief that the boxes were placed by Omega doesn’t pay rent any differently than the belief that the boxes just appeared at random in 100 locations so far. If we are to anticipate anything different knowing it was Omega’s doing, on what grounds? It could only be because we were distracted by vague notions about what Omega might be able to do or predict.
The following seemingly critical detail is just more misdirection and adds nothing either:
And the twist is that Omega has put a million dollars in box B iff Omega has predicted that you will take only box B.
I anticipate nothing differently whether this part is included or not, because nothing concrete is implied about Omega’s predictive powers—only “superintelligence from another galaxy,” which certainly sounds awe-inspiring but doesn’t tell me anything really useful (how hard is predicting my actions, and how super is “super”?).
The only detail that pays any rent is the one above in bold. Eliezer is right that one-boxing wins, but all you need to figure that out is Bayes.
Newcomb’s Problem is silly. It’s only controversial because it’s dressed up in wooey vagueness. In the end it’s just a simple probability question and I’m surprised it’s even taken seriously here. To see why, keep your eyes on the bolded text:
The problem is, such emphatic declarations of confidence in the right answer can just as easily be followed by one-boxing, two-boxing, or declaring the hypotheses self-contradictory. That is, in fact, what makes it a Problem, even if, to any individual, it is not a problem.
Differing outcomes are a problem by themselve. Either one reasoning is right and the others are wrong, or basic logic is broken (and it would follow all maths are broken). It could also be that some hypothesis absolutely necessary for one reasoning or the other are implicit and untelled.
This is why, even if to me Newcomb is not a problem, it is still critical to find where other’s reasoning are either broken or which assumptions are hidden. Failure to exhibit any error in someone else reasoning would lead to conclude that either my reasoning is broken (and I would have to find why) or that maths are broken. And I take that very seriously.
That’s also why when rejecting someone else reasoning stating we believe another well known reasoning is right (authority argument) is never enough. For the sake of rationality we should also find the error (if any) in the other’s reasoning.
To ask that question is already to presuppose the one-boxing answer, and to miss the problem that the problem itself may be problematic. I don’t take simple two-boxing any more seriously than Amanojack does, but the third possibility, of disputing that the problem is well-posed, is worth exploring. On LW, self-professed two-boxers are usually taking that alternative. (Elsewhere, I see two-boxing philosophers actually saying that two-boxing loses, but is still the rational thing to do.)
The problem is best disputed not by simply asserting, as some have, that no such Omega can exist, but by thinking in detail about what it would take for someone to predict the decisions of a decision-maker who knows you’re trying to predict their decisions. What that sort of thinking looks like is this. That paper is about Prisoners Dilemma, but similar investigations could be made of Newcomb, Parfit’s Hitchhiker, etc.
That is what fighting the hypothesis looks like, done right.
So, what is wrong believing in probabilities ?
To ask that question is already to presuppose the one-boxing answer, and to miss the problem that the problem itself may be problematic.
That is going for the third option and dodging to point out exactly why the problem should not be well posed. I can write a program working as the Newcomb’s problem is described if I go for the “unperfect predictor” version where the being is merely right “most of the time”. A way to do it could be to let player run a number of practice (or calibration) games, then at a time chosen by the guesser make that game “real”. The calibration plays would simulate the supernatural player minute observation of the player behavior, what can indeed not easily be done.
I knew of the Robust Coopearation paper, and it’s really very interresting, but getting the source code of the other is also a huge change to the initial problem. At least it excludes perfect oracles from the problem, it is also clear you may be confronted to halting problem (this is why current scheme tournament based on this idea had to make a provision in rules to avoid non halting programs). Stating we can say something usefull on another problem does not implies the initial one had anything wrong.
On the other hand, it is obvious that Dominance Argument is broken in Newcom’s problem (and also in PD) as the logical proof is only correct when we have non correlated variables (non correlation should not be confused with causal independance, causal independance is not enough for Dominance Argument to be correct). In Newcomb’s problem, the perfect correlation is part of the problem statement. How anyone could then apply Dominance Argument is beyond me, probably because it mimics usual deductive logic.
I’m not saying that Newcomb’s problem describe any physically possible event, or not even that it is a good problem, or that the consequences it leads to are agreeable (at first sight it leads to lack of free will), but just that mathematically using (very) simple probabilistic tools you can solve it, without changing anything and that alternative usual solution is based on a mathematical error.
I can write a program working as the Newcomb’s problem is described if I go for the “unperfect predictor” version where the being is merely right “most of the time”. A way to do it could be to let player run a number of practice (or calibration) games, then at a time chosen by the guesser make that game “real”. The calibration plays would simulate the supernatural player minute observation of the player behavior, what can indeed not easily be done.
Are you claiming to be able to write a program that can win, playing Omega in an iterated Newcomb’s Problem contest, basing its predictions only on the record of the outcomes of past training rounds? How high a prediction accuracy do you expect to be able to get?
As proposed, the idea is to run the program in “test mode”. To simulate the super-being Omenga we give it the opportunity to decide when game stops being a simulation (predictor calibration) and start being the “real game”. To be fair, this change (or the rules governing it) will be communicated to some external judge before the actual “real play”. But it will not be communicated to player (or obviously it would break any calibration accuracy). A possible rule could be to start the real game when some fixed accuracy is reached (something like prediction is right 99% of the time), or it could also be a fixed number of calibration games.
Writing such predictor is as easy as it can be : just a simple loop waiting for the fixed accuration wich is reached when either one-box or two-box is above expected threshold. Obviously it player is random, that could be quite long (but even with random it should happen sometime). But with a rational player that should be fast enough. I’m not sure that a random player could qualify as rational, anyways.
Doing that Omega can be as accurate as wished.
It still is not a perfect predictor, the player could still outguess Omega and predict at wich move the desired accuracy will be reached, but it’s good enough for me (and the Omega player could add some randomness on his side tu avoid guessers).
I see no reason why the program describe above could not be seen as an acceptable Omega following Newcomb’s problem rules.
Not communicating the actual real game is just here to avoid cheaters and enforce that the actual experiment will be done in the same environment sa the calibration.
I wonder if anyone would seriously choose to two-box any time with the above rules.
To be fair, this change (or the rules governing it) will be communicated to some external judge before the actual “real play”. But it will not be communicated to player (or obviously it would break any calibration accuracy).
But then, the player never knows when they are faced with Omega, the successful predictor, which is an essential part of Newcomb’s problem.
A possible rule could be to start the real game when some fixed accuracy is reached (something like prediction is right 99% of the time), or it could also be a fixed number of calibration games.
Writing such predictor is as easy as it can be : just a simple loop waiting for the fixed accuration wich is reached when either one-box or two-box is above expected threshold. Obviously it player is random, that could be quite long (but even with random it should happen sometime).
You expect to predict even a random choice with 99% accuracy? Am I misunderstanding something? Rock-scissors-paper programs that try to detect the non-randomness of human choices do succeed against most people, but only a little better than chance, not with 99% accuracy. Against a truly random player they do not succeed at all.
But iterated Newcomb is different from original Newcomb, just as iterated PD is different from plain PD. Now, I don’t see anything wrong with studying related problems, but you yourself said that studying a different but related problem does not touch the original.
I don’t know if you have seen it, but I have posted an actual program playing Newcomb’s game. As far as I understand what I have done, this is not an Iterated Newcomb’s problem, but a single shot one. You should also notice that the calibration phase does not returns output to the player (well, I added some showing of reached accuracy, but this is not necessary).
If I didn’t overviewed some detail, the predictor accuracy is currently tuned at above 90% but any level of accuracy is reachable.
As I explained yesterday, the key point was to run some “calibration” phase before running the actual game. To make the calibration usefull I have to blur the limit between calibration and actual game or the player won’t behave as in real game while in calibration phase. Hence the program need to run a number of “maybe real” games before playing the true one. For the reason explained above we also cannot say to the user he his playing the real and last game (or he would known if he is playing a calibration game or a real one and the calibration would be useless).
But it is very clear reading source code that if the (human) player was some kind of supernatural being he could defeat the program by choosing two boxes while the prediction is one-box. It just will be a very unlikely event to the desired accuracy level.
I pretend this is a true unmodified Newcomb’s problem, all the calibration process is here only to make actually true the preassertion of the Newcomb’s problem : prediction accuracy of Omega (and verifiably so for the human player : he can read the source code and convince himself or even run the program and understand why prediction will be accurate).
As I know it Necomb’s problem does not impose the way the initial preassertion of accuracy is reached. As programming goes, I’m merely composing two functions, the first one ensuring the entry preassertion of good prediction accuracy is true.
I see a problem with the proposed method. Your program learns how often, on average, its opponent one-boxes or two-boxes. If I (as Omega) learn that someone is a one-boxer, then I can predict that they will one-box next time, put money in box B, and be proved right. But then, in an iterated game, if the one-boxer learns that I am not predicting his decision in the individual case, but have made a general prediction once and for all and thereafter always filling box B, then he can with impunity take both boxes and prove my prediction wrong.
A true Omega needs to make both P(box B full | take one box) and P(box B empty | take both boxes) high. The proposed scheme ensures that P(box B full | habitual one-boxer) and P(box B empty | habitual two-boxer) are high, which is not quite the same.
Similarly, suppose I convince Eliezer that I’m Omega. He has publicly avowed one-boxing on Newcomb, so I can skip the learning phase, fill box B, and be proved right. But if, for some reason, he suspects that I’m not a superintelligent superbeing with superpowers of prediction, and in a series of games, experiments with two-boxing, I will be exposed as an impostor.
Iterated Newcomb played between programs given access to each other’s source code would be an interesting challenge. I assume Omega doesn’t care about the money, but plays for the gratification of correctly predicting the other player’s choice. The other player is playing for the money.
A simpler, zero-sum game also suggests itself to me. This is more like Rock-Paper-Scissors than Newcomb, but again the point is to play using knowledge of the other person’s code. Each player chooses 0 or 1. Player A wins if the choices are the same, player B wins if they are different.
(This might look as if A is trying to predict B and B is trying to avoid being predicted, but the game is actually symmetric, both players doing both of these things. Swap the labels on B’s choices and B wins on equality and A on inequality.)
In classical game theory, the optimal strategy is to toss a coin, and the expected payoff is zero. The challenge is to do better against real opponents.
A true Omega needs to make both P(box B full | take one box) and P(box B empty | take both boxes) high. The proposed scheme ensures that P(box B full | habitual one-boxer) and P(box B empty | habitual two-boxer) are high, which is not quite the same.
If I understand correctly the distinction you’re making between habitual one boxer and take one box the first kind would be about the past player history and the other one about the future. If so I guess you are right. I’m indeed using the past to make my prediction, as using the future is beyond my reach.
But I believe you’re missing the point. My program is not an iterated Newcomb’s Problem because Omega does not perform any prediction along the way. It will only perform one prediction. And that will be for the last game and the human won’t be warned. It does not care at all about the reputation of the player, but only on it’s acts in situations where he (the human player) can’t know if he is playing of not.
But another point of view is possible, and that is what comes to mind when you run the program: it is coercing the player to be either a one boxer or a two boxer if he wan’t to play at all. Any two-boxing and the player will have to spend a very long time one-boxing to reach the state when he is again seen as a one boxer. As it is written, the program is likely (to the chosen accuracy level) to make it’s prediction while the player is struggling to be a one boxer.
As a human player what comes through my mind while running my program is ok: I want to get a million dollars, henceforth I have to become a one boxer.
If my program runs as long as wished accuracy is nor reached it can reach any accuracy. Truly random numbers are also expected to deviate toward extremes sometimes in the long run (if they do not behave like that they are not random). As it is very rare events, against random players the expected accuracy would certainly never be reached in a human life.
Why I claim is the “calibration phase” described above takes place before Newcomb’s problem. When the actual game starts the situation described in Newcomb’s problem is exactly what is reached. THe description of the calibration phase could even be provided to the player to convince him Omega prediction will be accurate. At least it is convincing for me and in such a setting I would certaily believe Omega can predict my behavior. In a way you could the my calibration phase as a way for Omega to wait for the player to be ready to play truly instead of trying to cheat. As trying to cheat will only result in delaying the actual play.
OK. It may be another problem, what I did is merely replacing a perfectly accurate being with an infinitely patient one… but this one is easy to program.
I posted a possible program doing what I describe in another comment. The trick as expected is that it’s easier to change the human player understanding of the nature of omega to reach the desired predictability. In other words : you just remove human free will (and running my program the player learn very quickly that is in his best interrest), then you play. What is interresting is that the only way compatible with Newcomb’s problem description to remove his free will is to make it a one-boxer. The incentive to make it a two-boxer would be to exhibit a bad predictor and that’s not compatible with Newcomb’s problem.
A way to do it could be to let player run a number of practice (or calibration) games, then at a time chosen by the guesser make that game “real”. The calibration plays would simulate the supernatural player minute observation of the player behavior, what can indeed not easily be done.
No good, then even CDTers are incentivized to one-box, since once-boxing in the practice rounds causes higher rewards in the real rounds.
I do not see your reasoning here ? What I’m proposing is not letting know when practising round stops and real round starts. That means indeed that one boxer would get higher rewards in both practice and real round, and that’s why I believe it’s an argument for one boxing.
My proposal for “simulating” Newcomb’s may not be accurate (and it’s certainly not perfect) but you can’t conclude that based on the (projected) outcome of the experiment disagreeing with wath you expect.
Here is an actual program (written in python) implementing the described experiment. It has two stages. The first part is just calibration intending to find out if the player is one boxing or two boxing. The second is a straightforward non iterated Newcomb problem. Some randomness is used to avoid the player to exactly know when calibration stops and test begin, but calibration part does not care at all if it will predict the player is a one boxer or a two boxer it is just intended to create an actual predictor behaving as described in Newcomb’s.
print "I will run some trial games (at least 5) to calibrate the predictor."
print ("As soon as the predictor will reach the expected quality level\n"
"I will run the actual Newcomb game. Be warned you won't be\n"
"warned when calibration phase will end and actual game begin\n"
"this is intended to avoid any perturbation of predictor accuracy.\n")
# run some prelude games (to avoid computing averages on too small a set)
# then compute averages to reach the intended prediction quality
# inecting some randomness in prelude and precision quality avoid
# anybody (including program writer) to be certain of when
# calibration ends. This is to avoid providing to user data that
# will change it's behavior and defeats prediction accuracy.
import random
# 5 to 25 calibration move
prelude = (5 + random.random() * 20.0)
# 90% accuracy or better, and avoid infinite loop
# we do not tell how much better to avoid guessers
accuracy = 1.0 - (random.random() * 0.1) - 0.01
# postlude is the number of test games where desired accuracy must be kept
# before running the actual game
# postlude will be a random number between 1 and 5 to avoid players guessing
# on the exact play time when percent will change, this could give them some
# hint on the exact final game time. It is possible the current postlude
# can still be exploited to improve cheater chances above intended predictor
# values, but it's just here to get the idea... and besides outguessing omega
# the cheater is only doing so in the hope of getting 100 bucks.
# How much energy does that deserve ?
postlude = 0
one = total = two = 0
while ((total < prelude) and (int(postlude) != 1)):
a = raw_input ("1 - One-box, 2 - Two-boxes : ")
if not a in ['1', '2']: continue
if a == '1':
one += 1
else:
two += 1
total += 1
print "current accuracy is %d%%" % int(100.0 * max(two, one) / total)
if (max(two, one) * 1.0 < total * accuracy):
if postlude != 0 :
postlude -= 1
else:
postlude = 1 + random.random() * 5.0
else:
postlude = 0
# Now prediction accuracy is good enough, run actual Newcomb's game
# prediction is truly a prediction of the future
# nothing prevents the user to choose otherwise.
#print "This is the actual Newcomb game, but I won't say it"
prediction = 1 if one > two else 2
finished = False
while not finished:
a = raw_input ("1 - One-box, 2 - Two-boxes : ")
if a == '1':
if prediction == 1:
print "You win 1 000 000 dollars"
else:
print "You win zero dollars"
finished = True
elif a == '2':
if prediction == 1:
print "You win 1 000 100 dollars"
else:
print "You win 100 dollars"
finished = True
After reading some of the Quantum Mechanics sequence, I am more willing to believe in Omega’s omniscience. Quantum mechanics allows for multiple timelines leading to the same outcome to interfere and simply never happen, even if they would have been probable in classical mechanics. Perhaps all timelines leading to the outcome where one-boxing does not yield money happen to interfere. Even if you take a more literal interpretation of the problem statement, where it is your own mind that determines the box’s content, your mind is made of particles which could conceivably affect the universe’s configuration.
I have more or less the same point of view and applied it to non iterated prisonner’s dilemma (as Newcomb’s is merely half a Prisonner’s Dilemma as David Lewis suggested in an article, and on this I agree with him, but not on his conclusion).
What is at stakes here (in Newcomb’s or PD) may not be that easy to accept anyway. It’s probability and Bayes against causality. The doom loop in Newcomb’s (reasoning loop leading to loose 1 million, as I see it) is stating that The content of the boxes is already put when you play, henceforth you action won’t change anything. The quantum mechanical reasoning would go the other way: as long as you did’nt observe/interact with it it is merely a probability. You may even want to go futher than that: imagine that someone else see the content of the box, then see you choosing the predicted set of boxes. He will conclude you have no freewill, or something along theses lines. I understand that people puting freewill as a fact—not merely a belief that could be contradicted by experiment—and so reject unthinkingly the probabilist reasoning.
…You know that paper goes on to assert that the two problems are meaningfully different, such that it’s rational to both one-box in Newcomb’s Problem and chew gum in Solomon’s Problem, right?
It’s only controversial because it’s dressed up in wooey vagueness
I also happen to think that under-specification of this puzzle adds significantly to the controversy.
What the puzzle doesn’t tell us is the properties of the universe in which it is set. Namely, whether the universe permits future to influence the past, which I’ll refer to as “future peeking”.
(alternatively, whether the universe somehow allows someone within the universe to precisely simulate the future faster than it actually comes—a proposition I don’t believe is ever true in any universe defined mathematically).
This is important because if the future can’t influence the past, then it is known with absolute certainty that taking two boxes won’t possibly change what’s in them (this is, after all, a basic given of the universe). Whether Omega has predicted something before is completely irrelevant now that the boxes are placed.
Alas, we aren’t told what the universe is like. If that is intentionally part of the puzzle then the only way to solve it would be to enumerate all possible universes, assigning each one a probability of being ours based on all the available evidence, and essentially come up with a probability that “future peeking” is impossible in our universe. One would then apply simple arithmetic to calculate the expected winnings.
Unfortunately P(“future peeking allowed”) it’s one of those probabilities that is completely incalculable for any practical purpose. Thus if “no future peeking” isn’t a given, the best answer is “I don’t know if taking two boxes is best because there’s this one probability I can’t actually calculate in practice”.
whether the universe somehow allows someone within the universe to precisely simulate the future faster than it actually comes—a proposition I don’t believe is ever true in any universe defined mathematically
As near as I can tell, this depends on dubious assumptions about a mathematical universe. You appear to treat time as fundamental, and yet reject the possibility that reality (or the Matrix) simulates a certain outcome happening at a certain time, not before (as we’d expect if reality calculated the output of a time-dependent wavefunction).
In addition, you seem to assume that reality cares about the same aspects of the situation that interest Omega. Otherwise it seems clear that Omega could get an answer sooner by leaving out all the details which don’t affect the human-level outcome.
While I disagree that one-boxing still wins, I’m most interested in seeing the “no future peeking” and the actual Omega success rate being defined as givens. It’s important that I can rely on the 99.9% value, rather than wondering whether it is perhaps inferred from their past 100 correct predictions (which could, with a non-negligible probability, have been a fluke).
That does indeed seem like the standard version of Newcomb’s. (Though I don’t understand your last sentence, assuming “non-negligible” does not mean 1⁄2 to the power of 100.)
Can you spell out what you mean by “if” in this context? Because a lot of us are explicitly talking about the best algorithm to program into an AI.
I’m not sure I understand correctly, but let me phrase the question differently: what sort of confidence do we have in “99.9%” being an accurate value for Omega’s success rate?
From your previous comment I gather the confidence is absolute. This removes one complication while leaving the core of the paradox intact. I’m just pointing out that this isn’t very clear in the original specification of the paradox, and that clearing it up is useful.
To explain why it’s important, let me indeed think of an AI like hairyfigment suggested. Suppose someone says they have let 100 previous AIs flip a fair coin 100 times each and it came out heads every single time, because they have magic powers that make it so. This someone presents me video evidence of this feat.
If faced with this in the real world, an AI coded by me would still bet close to 50% on tails if offered to flip its own fair coin against this person, because I have strong evidence that this someone is a cheat, and their video evidence is fake. Just something I know from a huge amount of background information that was not explicitly part of this scenario.
However, when discussing such scenarios, it is sometimes useful to assume hypothetical scenarios unlike the real world. For example, we could state that this someone has actually performed the feat, and that there is absolutely no doubt about that. That’s impossible in our real world, but it’s useful for the sake of discussing bayesianism. Surely any bayesianist’s AI would expect heads with high probability in this hypothetical universe.
So, are we looking at “Omega in the real world where someone I don’t even know tells me they are really damn good at predicting the future”, or “Omega in some hypothetical world where they are actually known with absolute certainty to be really good at predicting the future”?
Omega has been correct on each of 100 observed occasions so far—everyone who took both boxes has found box B empty and received only a thousand dollars; everyone who took only box B has found B containing a million dollars.
Seems to me the language of this rules out faked video. And to explain it as a newsletter scam would, I think, require postulating 2^100 civilizations that have contact with Omega but not each other. Note that we already have some reason to believe that a powerful and rational observer could predict our actions early on.
I’ve reviewed the language of the original statement and it seems that the puzzle is set in essentially the real world with two major givens, i.e. facts in which you have 100% confidence.
Given #1: Omega was correct on the last 100 occurrences.
Given #2: Box B is already empty or already full.
There is no leeway left for quantum effects, or for your choice affecting in any way what’s in box B. You cannot make box B full by consciously choosing to one-box. The puzzle says so, after all.
If you read it like this, then I don’t see why you would possibly one-box. Given #2 already implies the solution. 100 successful predictions must have been achieved through a very low probability event, or a trick, e.g by offering the bet only to those people whose answer you can already predict, e.g. by reading their LessWrong posts.
If you don’t read it like this, then we’re back to the “gooey vagueness” problem, and I will once again insist that the puzzle needs to be fully defined before it can be attempted. For example, by removing both givens, and instead specifying exactly what you know about those past 100 occurrences. Were they definitely not done on plants? Was there sampling bias? Am I considering this puzzle as an outside observer, or am I imagining myself being part of that universe—in the latter case I have to put some doubt into everything, as I can be hallucinating. These things matter.
With such clarifications, the puzzle becomes a matter of your confidence in the past statistics vs. your confidence about the laws of physics precluding your choice from actually influencing what’s in box B.
Newcomb’s Problem is silly. It’s only controversial because it’s dressed up in wooey vagueness. In the end it’s just a simple probability question and I’m surprised it’s even taken seriously here. To see why, keep your eyes on the bolded text:
What can we anticipate from the bolded part? The only actionable belief we have at this point is that 100 out of 100 times, one-boxing made the one-boxer rich. The details that the boxes were placed by Omega and that Omega is a “superintelligence” add nothing. They merely confuse the matter by slipping in the vague connotation that Omega could be omniscient or something.
In fact, this Omega character is superfluous; the belief that the boxes were placed by Omega doesn’t pay rent any differently than the belief that the boxes just appeared at random in 100 locations so far. If we are to anticipate anything different knowing it was Omega’s doing, on what grounds? It could only be because we were distracted by vague notions about what Omega might be able to do or predict.
The following seemingly critical detail is just more misdirection and adds nothing either:
I anticipate nothing differently whether this part is included or not, because nothing concrete is implied about Omega’s predictive powers—only “superintelligence from another galaxy,” which certainly sounds awe-inspiring but doesn’t tell me anything really useful (how hard is predicting my actions, and how super is “super”?).
The only detail that pays any rent is the one above in bold. Eliezer is right that one-boxing wins, but all you need to figure that out is Bayes.
EDIT: Spelling
The problem is, such emphatic declarations of confidence in the right answer can just as easily be followed by one-boxing, two-boxing, or declaring the hypotheses self-contradictory. That is, in fact, what makes it a Problem, even if, to any individual, it is not a problem.
Differing outcomes are a problem by themselve. Either one reasoning is right and the others are wrong, or basic logic is broken (and it would follow all maths are broken). It could also be that some hypothesis absolutely necessary for one reasoning or the other are implicit and untelled.
This is why, even if to me Newcomb is not a problem, it is still critical to find where other’s reasoning are either broken or which assumptions are hidden. Failure to exhibit any error in someone else reasoning would lead to conclude that either my reasoning is broken (and I would have to find why) or that maths are broken. And I take that very seriously.
That’s also why when rejecting someone else reasoning stating we believe another well known reasoning is right (authority argument) is never enough. For the sake of rationality we should also find the error (if any) in the other’s reasoning.
So, what is wrong believing in probabilities ?
To ask that question is already to presuppose the one-boxing answer, and to miss the problem that the problem itself may be problematic. I don’t take simple two-boxing any more seriously than Amanojack does, but the third possibility, of disputing that the problem is well-posed, is worth exploring. On LW, self-professed two-boxers are usually taking that alternative. (Elsewhere, I see two-boxing philosophers actually saying that two-boxing loses, but is still the rational thing to do.)
The problem is best disputed not by simply asserting, as some have, that no such Omega can exist, but by thinking in detail about what it would take for someone to predict the decisions of a decision-maker who knows you’re trying to predict their decisions. What that sort of thinking looks like is this. That paper is about Prisoners Dilemma, but similar investigations could be made of Newcomb, Parfit’s Hitchhiker, etc.
That is what fighting the hypothesis looks like, done right.
That is going for the third option and dodging to point out exactly why the problem should not be well posed. I can write a program working as the Newcomb’s problem is described if I go for the “unperfect predictor” version where the being is merely right “most of the time”. A way to do it could be to let player run a number of practice (or calibration) games, then at a time chosen by the guesser make that game “real”. The calibration plays would simulate the supernatural player minute observation of the player behavior, what can indeed not easily be done.
I knew of the Robust Coopearation paper, and it’s really very interresting, but getting the source code of the other is also a huge change to the initial problem. At least it excludes perfect oracles from the problem, it is also clear you may be confronted to halting problem (this is why current scheme tournament based on this idea had to make a provision in rules to avoid non halting programs). Stating we can say something usefull on another problem does not implies the initial one had anything wrong.
On the other hand, it is obvious that Dominance Argument is broken in Newcom’s problem (and also in PD) as the logical proof is only correct when we have non correlated variables (non correlation should not be confused with causal independance, causal independance is not enough for Dominance Argument to be correct). In Newcomb’s problem, the perfect correlation is part of the problem statement. How anyone could then apply Dominance Argument is beyond me, probably because it mimics usual deductive logic.
I’m not saying that Newcomb’s problem describe any physically possible event, or not even that it is a good problem, or that the consequences it leads to are agreeable (at first sight it leads to lack of free will), but just that mathematically using (very) simple probabilistic tools you can solve it, without changing anything and that alternative usual solution is based on a mathematical error.
Are you claiming to be able to write a program that can win, playing Omega in an iterated Newcomb’s Problem contest, basing its predictions only on the record of the outcomes of past training rounds? How high a prediction accuracy do you expect to be able to get?
As proposed, the idea is to run the program in “test mode”. To simulate the super-being Omenga we give it the opportunity to decide when game stops being a simulation (predictor calibration) and start being the “real game”. To be fair, this change (or the rules governing it) will be communicated to some external judge before the actual “real play”. But it will not be communicated to player (or obviously it would break any calibration accuracy). A possible rule could be to start the real game when some fixed accuracy is reached (something like prediction is right 99% of the time), or it could also be a fixed number of calibration games.
Writing such predictor is as easy as it can be : just a simple loop waiting for the fixed accuration wich is reached when either one-box or two-box is above expected threshold. Obviously it player is random, that could be quite long (but even with random it should happen sometime). But with a rational player that should be fast enough. I’m not sure that a random player could qualify as rational, anyways.
Doing that Omega can be as accurate as wished.
It still is not a perfect predictor, the player could still outguess Omega and predict at wich move the desired accuracy will be reached, but it’s good enough for me (and the Omega player could add some randomness on his side tu avoid guessers).
I see no reason why the program describe above could not be seen as an acceptable Omega following Newcomb’s problem rules.
Not communicating the actual real game is just here to avoid cheaters and enforce that the actual experiment will be done in the same environment sa the calibration.
I wonder if anyone would seriously choose to two-box any time with the above rules.
But then, the player never knows when they are faced with Omega, the successful predictor, which is an essential part of Newcomb’s problem.
You expect to predict even a random choice with 99% accuracy? Am I misunderstanding something? Rock-scissors-paper programs that try to detect the non-randomness of human choices do succeed against most people, but only a little better than chance, not with 99% accuracy. Against a truly random player they do not succeed at all.
But iterated Newcomb is different from original Newcomb, just as iterated PD is different from plain PD. Now, I don’t see anything wrong with studying related problems, but you yourself said that studying a different but related problem does not touch the original.
I don’t know if you have seen it, but I have posted an actual program playing Newcomb’s game. As far as I understand what I have done, this is not an Iterated Newcomb’s problem, but a single shot one. You should also notice that the calibration phase does not returns output to the player (well, I added some showing of reached accuracy, but this is not necessary).
If I didn’t overviewed some detail, the predictor accuracy is currently tuned at above 90% but any level of accuracy is reachable.
As I explained yesterday, the key point was to run some “calibration” phase before running the actual game. To make the calibration usefull I have to blur the limit between calibration and actual game or the player won’t behave as in real game while in calibration phase. Hence the program need to run a number of “maybe real” games before playing the true one. For the reason explained above we also cannot say to the user he his playing the real and last game (or he would known if he is playing a calibration game or a real one and the calibration would be useless).
But it is very clear reading source code that if the (human) player was some kind of supernatural being he could defeat the program by choosing two boxes while the prediction is one-box. It just will be a very unlikely event to the desired accuracy level.
I pretend this is a true unmodified Newcomb’s problem, all the calibration process is here only to make actually true the preassertion of the Newcomb’s problem : prediction accuracy of Omega (and verifiably so for the human player : he can read the source code and convince himself or even run the program and understand why prediction will be accurate).
As I know it Necomb’s problem does not impose the way the initial preassertion of accuracy is reached. As programming goes, I’m merely composing two functions, the first one ensuring the entry preassertion of good prediction accuracy is true.
I see a problem with the proposed method. Your program learns how often, on average, its opponent one-boxes or two-boxes. If I (as Omega) learn that someone is a one-boxer, then I can predict that they will one-box next time, put money in box B, and be proved right. But then, in an iterated game, if the one-boxer learns that I am not predicting his decision in the individual case, but have made a general prediction once and for all and thereafter always filling box B, then he can with impunity take both boxes and prove my prediction wrong.
A true Omega needs to make both P(box B full | take one box) and P(box B empty | take both boxes) high. The proposed scheme ensures that P(box B full | habitual one-boxer) and P(box B empty | habitual two-boxer) are high, which is not quite the same.
Similarly, suppose I convince Eliezer that I’m Omega. He has publicly avowed one-boxing on Newcomb, so I can skip the learning phase, fill box B, and be proved right. But if, for some reason, he suspects that I’m not a superintelligent superbeing with superpowers of prediction, and in a series of games, experiments with two-boxing, I will be exposed as an impostor.
Iterated Newcomb played between programs given access to each other’s source code would be an interesting challenge. I assume Omega doesn’t care about the money, but plays for the gratification of correctly predicting the other player’s choice. The other player is playing for the money.
A simpler, zero-sum game also suggests itself to me. This is more like Rock-Paper-Scissors than Newcomb, but again the point is to play using knowledge of the other person’s code. Each player chooses 0 or 1. Player A wins if the choices are the same, player B wins if they are different.
(This might look as if A is trying to predict B and B is trying to avoid being predicted, but the game is actually symmetric, both players doing both of these things. Swap the labels on B’s choices and B wins on equality and A on inequality.)
In classical game theory, the optimal strategy is to toss a coin, and the expected payoff is zero. The challenge is to do better against real opponents.
If I understand correctly the distinction you’re making between habitual one boxer and take one box the first kind would be about the past player history and the other one about the future. If so I guess you are right. I’m indeed using the past to make my prediction, as using the future is beyond my reach.
But I believe you’re missing the point. My program is not an iterated Newcomb’s Problem because Omega does not perform any prediction along the way. It will only perform one prediction. And that will be for the last game and the human won’t be warned. It does not care at all about the reputation of the player, but only on it’s acts in situations where he (the human player) can’t know if he is playing of not.
But another point of view is possible, and that is what comes to mind when you run the program: it is coercing the player to be either a one boxer or a two boxer if he wan’t to play at all. Any two-boxing and the player will have to spend a very long time one-boxing to reach the state when he is again seen as a one boxer. As it is written, the program is likely (to the chosen accuracy level) to make it’s prediction while the player is struggling to be a one boxer.
As a human player what comes through my mind while running my program is ok: I want to get a million dollars, henceforth I have to become a one boxer.
If my program runs as long as wished accuracy is nor reached it can reach any accuracy. Truly random numbers are also expected to deviate toward extremes sometimes in the long run (if they do not behave like that they are not random). As it is very rare events, against random players the expected accuracy would certainly never be reached in a human life.
Why I claim is the “calibration phase” described above takes place before Newcomb’s problem. When the actual game starts the situation described in Newcomb’s problem is exactly what is reached. THe description of the calibration phase could even be provided to the player to convince him Omega prediction will be accurate. At least it is convincing for me and in such a setting I would certaily believe Omega can predict my behavior. In a way you could the my calibration phase as a way for Omega to wait for the player to be ready to play truly instead of trying to cheat. As trying to cheat will only result in delaying the actual play.
OK. It may be another problem, what I did is merely replacing a perfectly accurate being with an infinitely patient one… but this one is easy to program.
I posted a possible program doing what I describe in another comment. The trick as expected is that it’s easier to change the human player understanding of the nature of omega to reach the desired predictability. In other words : you just remove human free will (and running my program the player learn very quickly that is in his best interrest), then you play. What is interresting is that the only way compatible with Newcomb’s problem description to remove his free will is to make it a one-boxer. The incentive to make it a two-boxer would be to exhibit a bad predictor and that’s not compatible with Newcomb’s problem.
No good, then even CDTers are incentivized to one-box, since once-boxing in the practice rounds causes higher rewards in the real rounds.
I do not see your reasoning here ? What I’m proposing is not letting know when practising round stops and real round starts. That means indeed that one boxer would get higher rewards in both practice and real round, and that’s why I believe it’s an argument for one boxing.
My proposal for “simulating” Newcomb’s may not be accurate (and it’s certainly not perfect) but you can’t conclude that based on the (projected) outcome of the experiment disagreeing with wath you expect.
Because depending on the numbers in the setup your modified experiment doesn’t get at the disagreement between one-boxer and two-boxers.
Here is an actual program (written in python) implementing the described experiment. It has two stages. The first part is just calibration intending to find out if the player is one boxing or two boxing. The second is a straightforward non iterated Newcomb problem. Some randomness is used to avoid the player to exactly know when calibration stops and test begin, but calibration part does not care at all if it will predict the player is a one boxer or a two boxer it is just intended to create an actual predictor behaving as described in Newcomb’s.
I’m with you. You have to look at the outcomes, otherwise you end up running into the same logical blinders that make Quantum Mechanics hard to accept.
After reading some of the Quantum Mechanics sequence, I am more willing to believe in Omega’s omniscience. Quantum mechanics allows for multiple timelines leading to the same outcome to interfere and simply never happen, even if they would have been probable in classical mechanics. Perhaps all timelines leading to the outcome where
one-boxing does not yield moneyhappen to interfere. Even if you take a more literal interpretation of the problem statement, where it is your own mind that determines the box’s content, your mind is made of particles which could conceivably affect the universe’s configuration.I have more or less the same point of view and applied it to non iterated prisonner’s dilemma (as Newcomb’s is merely half a Prisonner’s Dilemma as David Lewis suggested in an article, and on this I agree with him, but not on his conclusion).
What is at stakes here (in Newcomb’s or PD) may not be that easy to accept anyway. It’s probability and Bayes against causality. The doom loop in Newcomb’s (reasoning loop leading to loose 1 million, as I see it) is stating that The content of the boxes is already put when you play, henceforth you action won’t change anything. The quantum mechanical reasoning would go the other way: as long as you did’nt observe/interact with it it is merely a probability. You may even want to go futher than that: imagine that someone else see the content of the box, then see you choosing the predicted set of boxes. He will conclude you have no freewill, or something along theses lines. I understand that people puting freewill as a fact—not merely a belief that could be contradicted by experiment—and so reject unthinkingly the probabilist reasoning.
My comment about PD is in this Sequence (http://lesswrong.com/lw/hl8/other_prespective_on_resolving_the_prisoners/). I merely applty probability rules. I’m interrested to know if you see any fault in it from a probabilist point of view.
Do you also choose not to chew gum in Eliezer’s version of Solomon’s Problem?
…You know that paper goes on to assert that the two problems are meaningfully different, such that it’s rational to both one-box in Newcomb’s Problem and chew gum in Solomon’s Problem, right?
Alternately: It’s entirely precise and well formed, far more so than just about every real life decision.
Is there something wrong with my argument above?
I also happen to think that under-specification of this puzzle adds significantly to the controversy.
What the puzzle doesn’t tell us is the properties of the universe in which it is set. Namely, whether the universe permits future to influence the past, which I’ll refer to as “future peeking”.
(alternatively, whether the universe somehow allows someone within the universe to precisely simulate the future faster than it actually comes—a proposition I don’t believe is ever true in any universe defined mathematically).
This is important because if the future can’t influence the past, then it is known with absolute certainty that taking two boxes won’t possibly change what’s in them (this is, after all, a basic given of the universe). Whether Omega has predicted something before is completely irrelevant now that the boxes are placed.
Alas, we aren’t told what the universe is like. If that is intentionally part of the puzzle then the only way to solve it would be to enumerate all possible universes, assigning each one a probability of being ours based on all the available evidence, and essentially come up with a probability that “future peeking” is impossible in our universe. One would then apply simple arithmetic to calculate the expected winnings.
Unfortunately P(“future peeking allowed”) it’s one of those probabilities that is completely incalculable for any practical purpose. Thus if “no future peeking” isn’t a given, the best answer is “I don’t know if taking two boxes is best because there’s this one probability I can’t actually calculate in practice”.
As near as I can tell, this depends on dubious assumptions about a mathematical universe. You appear to treat time as fundamental, and yet reject the possibility that reality (or the Matrix) simulates a certain outcome happening at a certain time, not before (as we’d expect if reality calculated the output of a time-dependent wavefunction).
In addition, you seem to assume that reality cares about the same aspects of the situation that interest Omega. Otherwise it seems clear that Omega could get an answer sooner by leaving out all the details which don’t affect the human-level outcome.
Assume no “future peeking” and Omega only correctly predicting people as difficult to predict as you with 99.9% probability. One-boxing still wins.
While I disagree that one-boxing still wins, I’m most interested in seeing the “no future peeking” and the actual Omega success rate being defined as givens. It’s important that I can rely on the 99.9% value, rather than wondering whether it is perhaps inferred from their past 100 correct predictions (which could, with a non-negligible probability, have been a fluke).
That does indeed seem like the standard version of Newcomb’s. (Though I don’t understand your last sentence, assuming “non-negligible” does not mean 1⁄2 to the power of 100.)
Can you spell out what you mean by “if” in this context? Because a lot of us are explicitly talking about the best algorithm to program into an AI.
Why is it important to you that the success rate be a frequentialist probability rather than just a bayesian one?
I’m not sure I understand correctly, but let me phrase the question differently: what sort of confidence do we have in “99.9%” being an accurate value for Omega’s success rate?
From your previous comment I gather the confidence is absolute. This removes one complication while leaving the core of the paradox intact. I’m just pointing out that this isn’t very clear in the original specification of the paradox, and that clearing it up is useful.
To explain why it’s important, let me indeed think of an AI like hairyfigment suggested. Suppose someone says they have let 100 previous AIs flip a fair coin 100 times each and it came out heads every single time, because they have magic powers that make it so. This someone presents me video evidence of this feat.
If faced with this in the real world, an AI coded by me would still bet close to 50% on tails if offered to flip its own fair coin against this person, because I have strong evidence that this someone is a cheat, and their video evidence is fake. Just something I know from a huge amount of background information that was not explicitly part of this scenario.
However, when discussing such scenarios, it is sometimes useful to assume hypothetical scenarios unlike the real world. For example, we could state that this someone has actually performed the feat, and that there is absolutely no doubt about that. That’s impossible in our real world, but it’s useful for the sake of discussing bayesianism. Surely any bayesianist’s AI would expect heads with high probability in this hypothetical universe.
So, are we looking at “Omega in the real world where someone I don’t even know tells me they are really damn good at predicting the future”, or “Omega in some hypothetical world where they are actually known with absolute certainty to be really good at predicting the future”?
Seems to me the language of this rules out faked video. And to explain it as a newsletter scam would, I think, require postulating 2^100 civilizations that have contact with Omega but not each other. Note that we already have some reason to believe that a powerful and rational observer could predict our actions early on.
So you tell me what we should expect here.
I’ve reviewed the language of the original statement and it seems that the puzzle is set in essentially the real world with two major givens, i.e. facts in which you have 100% confidence.
Given #1: Omega was correct on the last 100 occurrences.
Given #2: Box B is already empty or already full.
There is no leeway left for quantum effects, or for your choice affecting in any way what’s in box B. You cannot make box B full by consciously choosing to one-box. The puzzle says so, after all.
If you read it like this, then I don’t see why you would possibly one-box. Given #2 already implies the solution. 100 successful predictions must have been achieved through a very low probability event, or a trick, e.g by offering the bet only to those people whose answer you can already predict, e.g. by reading their LessWrong posts.
If you don’t read it like this, then we’re back to the “gooey vagueness” problem, and I will once again insist that the puzzle needs to be fully defined before it can be attempted. For example, by removing both givens, and instead specifying exactly what you know about those past 100 occurrences. Were they definitely not done on plants? Was there sampling bias? Am I considering this puzzle as an outside observer, or am I imagining myself being part of that universe—in the latter case I have to put some doubt into everything, as I can be hallucinating. These things matter.
With such clarifications, the puzzle becomes a matter of your confidence in the past statistics vs. your confidence about the laws of physics precluding your choice from actually influencing what’s in box B.