In the sense of the language of “where U is a universal (monotone Turing) machine executing q given a1..am.”. (I unilaterally resorted to using same language as Hutter to minimize confusion—seems like a reasonable thing for two argues to adopt...).
I think you’re misunderstanding the language here; the inputs of a Turing machine are not part of the Turing machine. The program “q” and the actions “a1...am” are both inputs to U. That said, I guess it doesn’t matter if you have two different Us, because they will always compute the same output given the same inputs anyway.
In AIXI’s hypothetical where it takes 1 box, it taking 1 box is caused by the initial state of the Turing machine. It literally sets up the initial state of the TM so that it ends up picking 1 box (by the way of putting it’s picking 1 box on an extra tape, or appending it at the end of the program). Not by a later intervention, which would make no sense and be folk physics that’s wrong. The prediction, likewise, can be caused by the initial state.
According to AIXI’s assumption of “chronological Turing machines”, this isn’t quite right. If the bit was simply encoded into the “initial state” of the universe, then AIXI could potentially observe the state of that bit before it actually takes its action. Any models where that bit influences AIXI’s observations prior to acting would directly violate Hutter’s assumption; world programs of this kind are explicitly forbidden from occupying any of AIXI’s probability mass.
Now, I’ll grant that this is subtly different from an assumption of “forward causality” because the chronological assumption specifically applies to AIXI’s subjective past, rather than the actual past. However, I would argue that models in which AIXI’s action bit causes the contents of the box without affecting its past observations would necessarily be more complex. In order for such a model to satisfy AIXI’s assumptions, the action bit needs to do one of two things: 1) Magic (i.e. makes stuff appear or disappear within the box). 2) The action bit would need to be entangled with the state of the universe in just the right way; it would just so happen that Omega can observe that action bit but AIXI cannot observe it until after it takes its action.
It seems to me that Solomonoff induction will penalise both kinds of “action-caused” models quite heavily, because they offer a poor description of the problem. If the action bit was truly part of the state of the universe, it seems rather unlikely that Omega would be able to observe it while AIXI would not.
Re: your set up.
If the agents are complicated and fairly opaque (if they aren’t its not necessarily reasonable to assume what predictor does for them would be what predictor does for a complicated and opaque agent), and if agents are one or two boxing basically at an uniform random (AIXI won’t learn much if they all 2-box), [...]
I wouldn’t assume the agents are one-boxing or two-boxing at uniform random, that would be pretty stupid since Omega would be unable to predict them. Typical versions of Newcomb’s problem stipulate that when Omega thinks you’ll pick randomly it won’t put the million dollars in. Rather, it would be better to say that the agents are picked from some pool of agents, and it turns out that AIXI gets to witness reasonable proportions of both two-boxers and one-boxers.
[...] there’s a string with thousand ones and zeroes, and it’s repetition, which gives up to a 2^-1000 penalty to the representations where those are independently encoded using 2000 bits rather than 1000 bits.
Surely AIXI will compress those two same bitstrings into one bitstring somehow.
I completely agree! Given the enormous 1000-bit penalty, AIXI should determine that the problem is quite well described by a “common cause” explanation—that is, the actions of the individual actions and Omega’s prediction are both determined in advance by the same factors.
In fact, I would go even further than that; AIXI should be able to duplicate Omega’s feat and quickly come up with a universe model that predicts the agents as well as or better than Omega. When AIXI observed Omega playing the game it had access to the same information about the agents that Omega did, and so whatever the source of Omega’s predictive accuracy, AIXI should be able to replicate it.
More generally, I would argue a “common cause” explanation is implicit in Newcomb’s problem, and I think that AIXI should be able to deduce reasonable models of this without making such direct observations.
In any case, once AIXI comes upon this kind of explanation (which I think is really implicit in the setup of Newcomb’s problem), AIXI is doomed. Models in which AIXI’s future action bit manages to be observable to Omega without being observable to AIXI will be significantly more complicated than models in which Omega’s prediction is determined by information that AIXI has already observed.
The most obvious such model is the one I suggested before—Omega simply reasons in a relatively abstract way about the AIXI equation itself. All of this information is information that is accessible to AIXI in advance, and hence it cannot be dependent upon the future action bit.
As such, AIXI should focus in on world models where the box already contains a million dollars, or already does not. Since AIXI will determine that it’s optimal to two-box in both kinds of world, AIXI will two-box, and since this is a pretty simple line of reasoning Omega will predict that AIXI will two-box, and hence AIXI gets $1000.
It seems to me that Solomonoff induction will penalise both kinds of “action-caused” models quite heavily, because they offer a poor description of the problem.
What’s quite heavily? It seems to me that you can do that in under a hundred bits, and few hundred bits of information are not that hard to acquire. If I throw a die, and it does a couple dozen bounces, it’s influenced by the thermal noise and quantum fluctuations, there’s about 2.58 bits of information that is new even to the true magical AIXI. There’s kilobits if not megabits that can be learned from e.g. my genome (even if AIXI sat for years watching a webcam and browsing the internet beforehand, it still won’t be able to predict the quantum random—it’s either non deterministic, or multiverse and you don’t know where in the multiverse you are). AIXI presumably got a webcam or other high throughput input device, too, so it may be able to privilege some hundreds bits penalized hypothesis (rule out all simpler ones) in a fraction of a second.
It strikes me as you are thinking up a fairly specific environment where the observations do not provide enough actual information. If I observe a software predictor predict a bunch of simple programs with source that I know and can fully evaluate myself, that wouldn’t come close to convincing me it is going to predict my human decision, either. And I’m fundamentally more similar to those programs than AIXI is to agents it can predict.
What’s about my example environment, where AIXI lives in a house, plays Newcomb’s problem many times, and sometimes wants to get a lot of money, and sometimes doesn’t want to get a lot of money for various reasons, e.g. out of e.g. a fear of increasing the risk of burglary if it has too much money, or the fear of going to jail on money laundering charges, or what ever. Every time, the presence of money in the first box is correlated with the actual decision AIXI makes. This has to go on for long enough, of course, until the inherently unlikely hypothesis of being predicted by something, gets privileged.
Why would AIXI privilege a magical hypothesis like that one when there are other hypotheses are strictly simpler and explain the world better? If Omega is capable of predicting AIXI reasonably accurately without the kind of magic you’re proposing, why would AIXI possibly come up with a magical explanation that involves Omega having some kind of privileged, back-in-time access to AIXI’s action which has absolutely no impact on AIXI’s prior observations!?
As for your example environment, Iterated Newcomb’s problem != Newcomb’s problem, and the problem isn’t even Newcomb’s problem to begin with if AIXI doesn’t always want the million dollars. As far as I can tell, though, you’re just trying to come up with a setup in which Omega really needs to be retrocausal or magical, rather than just really good at predicting.
In other words, AIXI’s action is predetermined by the AIXI equation. A model where Omega predicts AIXI’s action on the basis of the AIXI equation is strictly simpler than a model involving a single bit of information that is entangled with Omega and yet somehow doesn’t leak out into the universe and remains invisible to AIXI until AIXI opens the box (or boxes).
Unless the universe and/or Omega really is magical, AIXI’s observations should obviously favour the real explanation over the magical one.
and yet somehow doesn’t leak out into the universe and remains invisible to AIXI until AIXI opens the box (or boxes).
This is kind of stipulated in the problem, the box being opaque, no? What does this leak look like, other than box being in some way not opaque?
We could talk of Superman Decision Theory here and how it necessarily two boxes because he has x-ray vision and sees into the box :)
Unless the universe and/or Omega really is magical, AIXI’s observations should obviously favour the real explanation over the magical one.
You keep asserting this, but I don’t see why that’s true. Let’s suppose that I design something. I have a bit of quantum shot noise in my synapses, the precise way how I implement something probably takes a lot of information to describe. Kilobits, megabits even. Meanwhile, the body of AIXI’s robot is magical—it’s reading from the action tape to set voltages on some wires or some such. So there’s some code to replicate...
And with regards to having a real explanation available, I think it falls under the purview of box not being opaque enough. It’s akin to having a videotape of omega putting or not putting money into the box.
I think I see a better way to clarify my original remark. There is a pretty easy way to modify AIXI to do CDT. Exclude machines that read Ai before printing Oi . (And it’s very possible that some approximations or other variations by Hutter did that. I’d certainly do that if I were making an approximation. I’d need to re-read him to make totally sure he didn’t have that somewhere) I think we can both agree that if you don’t do this, you can one-box without having backwards in time causation in your model, unlike CDT (and if you do, you can’t, like CDT).
In any case, my main point is that the one boxing and two boxing depends to the way of doing physics, and given that we’re arguing about different environments yielding different ways of doing physics, I think we agree on that point.
edit: also I think I can write a variation of AIXI that matches my decisionmaking more closely. I’d just require the TM to print actions on a tape, matching the hypothetical actions. Up to a constant difference in program lengths, so it’s not worse than a choice of a TM. (I’d prefer that not to screw up probabilities though, even if its up to a constant, I need to think how edit: actually quite easy to renormalize that away… I’m far too busy right now with other stuff though). Also using some symbolic package to approximately evaluate it, evading some of can’t model oneself traps.
There could be an universe that used some ahead of time indirect evaluation to tell me in advance what action I am going to take, and with me not taking another action out of spite. I don’t know for sure our universe isn’t this—I just have a somewhat low prior for that.
This is kind of stipulated in the problem, the box being opaque, no? What does this leak look like, other than box being in some way not opaque?
No, “the box is opaque” is very different to “you have absolutely no way of working out whether the box contains the million dollars”. For example, if I’m playing Newcomb’s problem then no matter how opaque the box is I’m already pretty sure it contains a million dollars, and when I proceed to one-box every time I’m almost always right. Are you saying I’m forbidden from being able to play Newcomb’s problem?
We could talk of Superman Decision Theory here and how it necessarily two boxes because he has x-ray vision and sees into the box :)
If “Superman Decision Theory” sees into the box and necessarily two-boxes, then Superman Decision Theory is doing it wrong, because SDT is always going to get $1000 and not a million dollars.
You keep asserting this, but I don’t see why that’s true. Let’s suppose that I design something. I have a bit of quantum shot noise in my synapses, the precise way how I implement something probably takes a lot of information to describe. Kilobits, megabits even. Meanwhile, the body of AIXI’s robot is magical—it’s reading from the action tape to set voltages on some wires or some such.
AIXI doesn’t need to work out the precise way Omega is implemented, it just needs to find programs that appear to function the same way. If the quantum noise has no impact on Omega’s actual predictions then it’s not going to matter as far as AIXI’s hypotheses about how Omega predicts AIXI are concerned.
As far as I can see, “the box is not opaque enough” translates to “AIXI knows too much about the real world!” Well, how is it a good thing if your decision theory performs worse when it has a more realistic model of the world?
It seems to me that the more the AIXI agent works out about the world, the more physics-like its programs should become, and consequently the less it will be able to come up with the kinds of explanations in which its actions cause things in the past.
Yes, world programs which involve reverse causality that just happens to be unobservable to AIXI until after it takes its own action are permissible Turing machines for AIXI, but the more information AIXI gathers, the higher the complexity penalty on those kinds of programs will be.
Why? Because, by necessity, AIXI’s action bit would have to be treated as a special case. If Omega predicts all those other agents in the same way, a bit of code that says “ah, but if Omega is facing AIXI, then Omega should access the a_5 bit from AIXI’s action tape instead of doing what Omega normally does” is simply an unnecessary complexity penalty that doesn’t help to explain anything about AIXI’s past observations of Omega.
So, the more mature AIXI gets, the more CDT-like it becomes.
No, “the box is opaque” is very different to “you have absolutely no way of working out whether the box contains the million dollars”.
I take the box being opaque to mean that the contents of the box do not affect my sensory input, and by extension that I don’t get to e.g. watch a video of omega putting money in the box, or do some forensic equivalent.
For example, if I’m playing Newcomb’s problem then no matter how opaque the box is I’m already pretty sure it contains a million dollars
Really? What if Omega is a program, which you know predicts outputs of other simple programs written in C++, Java, and Python, and it been fed your raw DNA as a description, ’cause you’re human?
What if you just know the exact logic Omega is using?
(Besides, decision theories tend to agree that you should pretend online that you one-box)
If “Superman Decision Theory” sees into the box and necessarily two-boxes, then Superman Decision Theory is doing it wrong, because SDT is always going to get $1000 and not a million dollars.
No, you just adapt the Newcomb’s “opaque box” in an obtuse way. Superman’s facing an entirely different decision problem from the Newcomb’s that you face.
Why? Because, by necessity, AIXI’s action bit would have to be treated as a special case. If Omega predicts all those other agents in the same way, a bit of code that says “ah, but if Omega is facing AIXI, then Omega should access the a_5 bit from AIXI’s action tape instead of doing what Omega normally does” is simply an unnecessary complexity penalty that doesn’t help to explain anything about AIXI’s past observations of Omega.
I think you’re just describing a case where AIXI fails to learn anything from other agents because they’re too different from the AIXI. What’s about my scenario where AIXI plays Newcomb’s multiple times, sometimes wanting more money and sometimes not? The program reading a_5 also appears to work right.
It seems to me that the more the AIXI agent works out about the world, the more physics-like its programs should become, and consequently the less it will be able to come up with the kinds of explanations in which its actions cause things in the past.
Well, given that predictors for AIXI are non existent, that should be the case.
edit: actually, what’s your reasons for one-boxing?
edit2: also I think this way of seeing the world—where your actions are entirely unlinked to the past—is a western phenomenon, some free will philosophy stuff. A quarter of my cultural background is quite fatalist in the outlook, so I see my decisions as the consequences of the laws of physics acting on the initial world state, and given same ‘random noise’, different decision by me implies both different future and different past.
I take the box being opaque to mean that the contents of the box do not affect my sensory input,
Yep, that’s what the box being opaque means—the contents of the box have no causal effect on your perceptions.
and by extension that I don’t get to e.g. watch a video of omega putting money in the box, or do some forensic equivalent.
Nope. Watching the video would contradict this principle as well, because you would still effectively be seeing the contents of the box.
What IS allowed by Newcomb’s problem, however, is coming to the conclusion that the contents of the box and your perceptions of Omega have a common cause in terms of how Omega functions or acts. You are then free to use that reasoning to work out what the contents of the box could be.
Your interpretation of Newcomb’s problem basically makes it incoherent. For example, let’s say I’m a CDT agent and I believe Omega predicted me correctly. Then, at the moment I make my decision to two-box, but before I actually see the contents of the opaque box, I already know that the opaque box is empty. Does this mean that the box is not “opaque”, by your reasoning?
Really? What if Omega is a program, which you know predicts outputs of other simple programs written in C++, Java, and Python, and it been fed your raw DNA as a description, ’cause you’re human?
If I don’t think Omega is able to predict me, then it’s not Newcomb’s problem, is it? Even if we assume that the Omega program is capable of predicting humans, DNA is not that likely to be sufficient evidence for it to be able to make good predictions.
What if you just know the exact logic Omega is using?
Well, then it obviously depends on what that exact logic is.
I think you’re just describing a case where AIXI fails to learn anything from other agents because they’re too different from the AIXI. What’s about my scenario where AIXI plays Newcomb’s multiple times, sometimes wanting more money and sometimes not? The program reading a_5 also appears to work right.
First of all, as I said previously, if AIXI doesn’t want the money then the scenario is not Newcomb’s. Also, I don’t think the a_5 reading program will end up being the simplest explanation even in that scenario. The program would need to use something like a_5, a_67, a_166, a_190 and a_222 in each instance of Newcomb’s problem respectively. Rather than a world program with a generic “get inputs from AIXI” subroutine, you need a world program with a “recognize Newcomblike problems and use the appropriate bits” subroutine; there is still a complexity penalty.
Unless you’re trying to make a setup in which Omega necessarily works by magic, then given sufficient evidence of reality at large magic is always going to be penalised. Given that reality at large works in a non-magical way, explanations that bootstrap your model of reality at large are always going to be simpler than explanations that have to add extraneous elements of “magic” to the model.
Besides, if Omega is just plain magical, then Newcomb’s problem boils down to “is a million bigger than a thousand?”
Well, given that predictors for AIXI are non existent, that should be the case.
Of course there can be predictors for AIXI. I can, for example, predict with a high degree of confidence that if AIXI knows what chess is and it wants to beat me at chess, it’s going to beat me. Also, if AIXI wants to maximise paperclips, I can easily predict that there are going to be a lot of paperclips.
edit: actually, what’s your reasons for one-boxing?
By being the kind of person who one-boxes, I end up with a million dollars instead of a thousand.
edit2: also I think this way of seeing the world—where your actions are entirely unlinked to the past—is a western phenomenon, some free will philosophy stuff. A quarter of my cultural background is quite fatalist in the outlook, so I see my decisions as the consequences of the laws of physics acting on the initial world state, and given same ‘random noise’, different decision by me implies both different future and different past.
Um, the “libertarian free will” perspective is mostly what I’m arguing against here. The whole problem with CDT is that it takes that perspective, and, in concluding that its action is not in any way caused by its past, it ends up with only $1000. My point is that AIXI ultimately suffers from the same problem; it assumes that it has this magical kind of free will when it actually does not, and also ends up with $1000.
Yep, that’s what the box being opaque means—the contents of the box have no causal effect on your perceptions.
Yeah, and then you kept stipulating that the model where Omega has read the action tape and then put or not put money into the box, but it didn’t leak onto sensory input, is very unlikely, and I noted that it’s stipulated in the problem statement that the box contents do not leak onto sensory input.
My point is that AIXI ultimately suffers from the same problem; it assumes that it has this magical kind of free will when it actually does not, and also ends up with $1000.
Let’s say AIXI lives inside the robot named Alice. According to every model employed by AIXI, the robot named Alice has pre-committed, since the beginning of time, to act out a specific sequence of actions. How the hell that assumes magical free will I don’t know. edit: and note that you can exclude machines which had read the action before printing matching sensory data, to actually ensure magical free will. I’m not even sure, maybe some variations by Hutter do just that.
edit:
Of course there can be predictors for AIXI. I can, for example, predict with a high degree of confidence that if AIXI knows what chess is and it wants to beat me at chess, it’s going to beat me. Also, if AIXI wants to maximise paperclips, I can easily predict that there are going to be a lot of paperclips.
That’s just abstruse. We both know what I mean.
By being the kind of person who one-boxes, I end up with a million dollars instead of a thousand.
Well, you’re just pre-committed to 1-box, then. The omegas that don’t know you’re pre-committed to 1-box (e.g. don’t trust you, can’t read your pre-committments, etc) would put nothing there, though, which you might be motivated to think about if its e.g. 10 millions vs 1 million, or 2 millions vs 1 million. (I wonder if one boxing is dependent on inflation...)
edit: let’s say I am playing the omega, and you know I know this weird trick for predicting you on the cheap.… you can work out what’s in the first box, can’t you? If you want money and don’t care of proving omega wrong out of spite, I can simply put nothing in the first box, and count on you to figure that out. You might have committed to the situation with 1 million vs 1 thousand, but I doubt you committed to 1000000 vs 999999 . You say you one box, fine, you get nothing—a rare time Omega is wrong.
edit2: a way to actually do Newcomb’s in real life, by the way. Take poor but not completely stupid people, make it 1000 000 vs 999 999 , and you can be almost always right. You can also draw some really rich people who you believe don’t really care and would 1-box for fun, and put a million in the first box for those, and be almost always right about both types of the case.
Yeah, and then you kept stipulating that the model where Omega has read the action tape and then put or not put money into the box, but it didn’t leak onto sensory input, is very unlikely, and I noted that it’s stipulated in the problem statement that the box contents do not leak onto sensory input.
The two situations are quite different. Any complexity penalty for the non-leaking box has already been paid via AIXI’s observations of the box and the whole Newcomb’s setup; the opaqueness of the box just boils down to normal reality.
On the other hand, your “action bit” model in which Omega reads AIXI’s action tape is associated with a significant complexity penalty because of the privileged nature of the situation—why specifically Omega, and not anyone else? Why does Omega specifically access that one bit, and not one of the other bits?
The more physics-like and real AIXI’s Turing machines get, the more of a penalty will be associated with Turing machines that need to incorporate a special case for a specific event.
Let’s say AIXI lives inside the robot named Alice. According to every model employed by AIXI, the robot named Alice has pre-committed, since the beginning of time, to act out a specific sequence of actions. How the hell that assumes magical free will I don’t know.
edit: and note that you can exclude machines which had read the action before printing matching sensory data, to actually ensure magical free will. I’m not even sure, maybe some variations by Hutter do just that.
AIXI as defined by Hutter (not just some “variation”) has a foundational assumption that an action at time t cannot influence AIXI’s perceptions at times 1..t-1. This is entirely incompatible with a model of Alice where she has pre-commited since the beginning of time, because such an Alice would be able to discover her own pre-commitment before she took the action in question. AIXI, on the other hand, explicitly forbids world models where that can happen.
That’s just abstruse. We both know what I mean.
No, I don’t. My point is that although you can’t predict AIXI in the general case, there are still many cases where AIXI can be predicted with relative ease. My argument is still that Newcomb’s problem is one of those cases (and that AIXI two-boxes).
As for all of your scenarios with different Omegas or different amounts of money, obviously a major factor is how accurate I think Omega’s predictions are. If ze has only been wrong one time in a million, and this includes people who have been one-boxing as well, why should I spend much time thinking about the possibility that I could be the one time he gets it wrong?
Similarly, if you’re playing Omega and you don’t have a past history of correctly predicting one-boxing vs one-boxers, then yes, I two-box. However, that scenario isn’t Newcomb’s problem. For it to be Newcomb’s problem, Omega has to have a history of correctly predicting one-boxers as well as two-boxers.
I suspect the unspecified implementation of Omega hides assumptions if not contradictions. Let me propose a more concrete version:
The problem is presented by Conservative Finite Omega (CFO), who works by pulling the agent’s source code, simulating it for a long but finite time, and putting $1,000,000 in the opaque box iff the simulation is determined to definitely one-box. The agent never walks away with the full $1,001,000, though the agent does sometimes walk away with $0.
So, assuming AIXI is confident in accurate models of how CFO works, CFO will simulate AIXI, which requires it to simulate AIXI’s (accurate) simulation of CFO—endless recursion. AIXI ‘wins’ the timeout war (correctly predicting CFO’s timeout), concludes that CFO has left the opaque box empty, and two-boxes.
You could look at that outcome as AIXI being penalized for being too smart. You could also say that an even smarter agent would prepend ‘if (facing CFO-like situation) then return one-box’ to its source code. Fundamentally, the specification of AIXI cannot conceive of its source code being an output; it’s baked into the assumptions that the explicit output bits are the only outputs.
Sure, I don’t necessarily blame the AIXI equation when it’s facing a relatively stupid Omega in that kind of situation.
However, consider “More Intelligent Finite Omega”, who pulls the agent’s source code and uses an approximate theorem-proving approach until it determines, with high confidence, what AIXI is going to do. Assuming that AIXI has received sufficient evidence to be reasonably confident in its model of MIFO, MIFO can reason like this:
AIXI will be able to accurately simulate me, therefore it will either have determined that the box is already empty, or already full.
Given either of those two models, AIXI will calculate that the best action is to two-box.
Consequently, AIXI will two-box. and then MIFO will leave the opaque box empty, and its prediction will have been correct. Moreover, MIFO had no other choice; if it were put the money in the opaque box, AIXI would still have two-boxed, and MIFO’s prediction would have been incorrect.
If you’re allowed to make the assumption that AIXI is confident in its model of CFO and CFO knows this, then I can make the same assumption about MIFO.
I think you’re right. At first I was worried (here and previously in the thread) that the proof that AIXI would two-box was circular, but I think it works out if you fill in the language about terminating turing machines and stuff. I was going to write up my formalization, but once I went through it in my head your proof suddenly looked too obviously correct to be worth expanding.
I think you’re misunderstanding the language here; the inputs of a Turing machine are not part of the Turing machine. The program “q” and the actions “a1...am” are both inputs to U. That said, I guess it doesn’t matter if you have two different Us, because they will always compute the same output given the same inputs anyway.
According to AIXI’s assumption of “chronological Turing machines”, this isn’t quite right. If the bit was simply encoded into the “initial state” of the universe, then AIXI could potentially observe the state of that bit before it actually takes its action. Any models where that bit influences AIXI’s observations prior to acting would directly violate Hutter’s assumption; world programs of this kind are explicitly forbidden from occupying any of AIXI’s probability mass.
Now, I’ll grant that this is subtly different from an assumption of “forward causality” because the chronological assumption specifically applies to AIXI’s subjective past, rather than the actual past. However, I would argue that models in which AIXI’s action bit causes the contents of the box without affecting its past observations would necessarily be more complex. In order for such a model to satisfy AIXI’s assumptions, the action bit needs to do one of two things:
1) Magic (i.e. makes stuff appear or disappear within the box).
2) The action bit would need to be entangled with the state of the universe in just the right way; it would just so happen that Omega can observe that action bit but AIXI cannot observe it until after it takes its action.
It seems to me that Solomonoff induction will penalise both kinds of “action-caused” models quite heavily, because they offer a poor description of the problem. If the action bit was truly part of the state of the universe, it seems rather unlikely that Omega would be able to observe it while AIXI would not.
I wouldn’t assume the agents are one-boxing or two-boxing at uniform random, that would be pretty stupid since Omega would be unable to predict them. Typical versions of Newcomb’s problem stipulate that when Omega thinks you’ll pick randomly it won’t put the million dollars in. Rather, it would be better to say that the agents are picked from some pool of agents, and it turns out that AIXI gets to witness reasonable proportions of both two-boxers and one-boxers.
I completely agree! Given the enormous 1000-bit penalty, AIXI should determine that the problem is quite well described by a “common cause” explanation—that is, the actions of the individual actions and Omega’s prediction are both determined in advance by the same factors.
In fact, I would go even further than that; AIXI should be able to duplicate Omega’s feat and quickly come up with a universe model that predicts the agents as well as or better than Omega. When AIXI observed Omega playing the game it had access to the same information about the agents that Omega did, and so whatever the source of Omega’s predictive accuracy, AIXI should be able to replicate it.
More generally, I would argue a “common cause” explanation is implicit in Newcomb’s problem, and I think that AIXI should be able to deduce reasonable models of this without making such direct observations.
In any case, once AIXI comes upon this kind of explanation (which I think is really implicit in the setup of Newcomb’s problem), AIXI is doomed. Models in which AIXI’s future action bit manages to be observable to Omega without being observable to AIXI will be significantly more complicated than models in which Omega’s prediction is determined by information that AIXI has already observed.
The most obvious such model is the one I suggested before—Omega simply reasons in a relatively abstract way about the AIXI equation itself. All of this information is information that is accessible to AIXI in advance, and hence it cannot be dependent upon the future action bit.
As such, AIXI should focus in on world models where the box already contains a million dollars, or already does not. Since AIXI will determine that it’s optimal to two-box in both kinds of world, AIXI will two-box, and since this is a pretty simple line of reasoning Omega will predict that AIXI will two-box, and hence AIXI gets $1000.
What’s quite heavily? It seems to me that you can do that in under a hundred bits, and few hundred bits of information are not that hard to acquire. If I throw a die, and it does a couple dozen bounces, it’s influenced by the thermal noise and quantum fluctuations, there’s about 2.58 bits of information that is new even to the true magical AIXI. There’s kilobits if not megabits that can be learned from e.g. my genome (even if AIXI sat for years watching a webcam and browsing the internet beforehand, it still won’t be able to predict the quantum random—it’s either non deterministic, or multiverse and you don’t know where in the multiverse you are). AIXI presumably got a webcam or other high throughput input device, too, so it may be able to privilege some hundreds bits penalized hypothesis (rule out all simpler ones) in a fraction of a second.
It strikes me as you are thinking up a fairly specific environment where the observations do not provide enough actual information. If I observe a software predictor predict a bunch of simple programs with source that I know and can fully evaluate myself, that wouldn’t come close to convincing me it is going to predict my human decision, either. And I’m fundamentally more similar to those programs than AIXI is to agents it can predict.
What’s about my example environment, where AIXI lives in a house, plays Newcomb’s problem many times, and sometimes wants to get a lot of money, and sometimes doesn’t want to get a lot of money for various reasons, e.g. out of e.g. a fear of increasing the risk of burglary if it has too much money, or the fear of going to jail on money laundering charges, or what ever. Every time, the presence of money in the first box is correlated with the actual decision AIXI makes. This has to go on for long enough, of course, until the inherently unlikely hypothesis of being predicted by something, gets privileged.
Why would AIXI privilege a magical hypothesis like that one when there are other hypotheses are strictly simpler and explain the world better? If Omega is capable of predicting AIXI reasonably accurately without the kind of magic you’re proposing, why would AIXI possibly come up with a magical explanation that involves Omega having some kind of privileged, back-in-time access to AIXI’s action which has absolutely no impact on AIXI’s prior observations!?
As for your example environment, Iterated Newcomb’s problem != Newcomb’s problem, and the problem isn’t even Newcomb’s problem to begin with if AIXI doesn’t always want the million dollars. As far as I can tell, though, you’re just trying to come up with a setup in which Omega really needs to be retrocausal or magical, rather than just really good at predicting.
In other words, AIXI’s action is predetermined by the AIXI equation. A model where Omega predicts AIXI’s action on the basis of the AIXI equation is strictly simpler than a model involving a single bit of information that is entangled with Omega and yet somehow doesn’t leak out into the universe and remains invisible to AIXI until AIXI opens the box (or boxes).
Unless the universe and/or Omega really is magical, AIXI’s observations should obviously favour the real explanation over the magical one.
This is kind of stipulated in the problem, the box being opaque, no? What does this leak look like, other than box being in some way not opaque?
We could talk of Superman Decision Theory here and how it necessarily two boxes because he has x-ray vision and sees into the box :)
You keep asserting this, but I don’t see why that’s true. Let’s suppose that I design something. I have a bit of quantum shot noise in my synapses, the precise way how I implement something probably takes a lot of information to describe. Kilobits, megabits even. Meanwhile, the body of AIXI’s robot is magical—it’s reading from the action tape to set voltages on some wires or some such. So there’s some code to replicate...
And with regards to having a real explanation available, I think it falls under the purview of box not being opaque enough. It’s akin to having a videotape of omega putting or not putting money into the box.
I think I see a better way to clarify my original remark. There is a pretty easy way to modify AIXI to do CDT. Exclude machines that read Ai before printing Oi . (And it’s very possible that some approximations or other variations by Hutter did that. I’d certainly do that if I were making an approximation. I’d need to re-read him to make totally sure he didn’t have that somewhere) I think we can both agree that if you don’t do this, you can one-box without having backwards in time causation in your model, unlike CDT (and if you do, you can’t, like CDT).
In any case, my main point is that the one boxing and two boxing depends to the way of doing physics, and given that we’re arguing about different environments yielding different ways of doing physics, I think we agree on that point.
edit: also I think I can write a variation of AIXI that matches my decisionmaking more closely. I’d just require the TM to print actions on a tape, matching the hypothetical actions. Up to a constant difference in program lengths, so it’s not worse than a choice of a TM. (I’d prefer that not to screw up probabilities though, even if its up to a constant, I need to think how edit: actually quite easy to renormalize that away… I’m far too busy right now with other stuff though). Also using some symbolic package to approximately evaluate it, evading some of can’t model oneself traps.
There could be an universe that used some ahead of time indirect evaluation to tell me in advance what action I am going to take, and with me not taking another action out of spite. I don’t know for sure our universe isn’t this—I just have a somewhat low prior for that.
No, “the box is opaque” is very different to “you have absolutely no way of working out whether the box contains the million dollars”. For example, if I’m playing Newcomb’s problem then no matter how opaque the box is I’m already pretty sure it contains a million dollars, and when I proceed to one-box every time I’m almost always right. Are you saying I’m forbidden from being able to play Newcomb’s problem?
If “Superman Decision Theory” sees into the box and necessarily two-boxes, then Superman Decision Theory is doing it wrong, because SDT is always going to get $1000 and not a million dollars.
AIXI doesn’t need to work out the precise way Omega is implemented, it just needs to find programs that appear to function the same way. If the quantum noise has no impact on Omega’s actual predictions then it’s not going to matter as far as AIXI’s hypotheses about how Omega predicts AIXI are concerned.
As far as I can see, “the box is not opaque enough” translates to “AIXI knows too much about the real world!” Well, how is it a good thing if your decision theory performs worse when it has a more realistic model of the world?
It seems to me that the more the AIXI agent works out about the world, the more physics-like its programs should become, and consequently the less it will be able to come up with the kinds of explanations in which its actions cause things in the past. Yes, world programs which involve reverse causality that just happens to be unobservable to AIXI until after it takes its own action are permissible Turing machines for AIXI, but the more information AIXI gathers, the higher the complexity penalty on those kinds of programs will be.
Why? Because, by necessity, AIXI’s action bit would have to be treated as a special case. If Omega predicts all those other agents in the same way, a bit of code that says “ah, but if Omega is facing AIXI, then Omega should access the a_5 bit from AIXI’s action tape instead of doing what Omega normally does” is simply an unnecessary complexity penalty that doesn’t help to explain anything about AIXI’s past observations of Omega.
So, the more mature AIXI gets, the more CDT-like it becomes.
I take the box being opaque to mean that the contents of the box do not affect my sensory input, and by extension that I don’t get to e.g. watch a video of omega putting money in the box, or do some forensic equivalent.
Really? What if Omega is a program, which you know predicts outputs of other simple programs written in C++, Java, and Python, and it been fed your raw DNA as a description, ’cause you’re human?
What if you just know the exact logic Omega is using?
(Besides, decision theories tend to agree that you should pretend online that you one-box)
No, you just adapt the Newcomb’s “opaque box” in an obtuse way. Superman’s facing an entirely different decision problem from the Newcomb’s that you face.
I think you’re just describing a case where AIXI fails to learn anything from other agents because they’re too different from the AIXI. What’s about my scenario where AIXI plays Newcomb’s multiple times, sometimes wanting more money and sometimes not? The program reading a_5 also appears to work right.
Well, given that predictors for AIXI are non existent, that should be the case.
edit: actually, what’s your reasons for one-boxing?
edit2: also I think this way of seeing the world—where your actions are entirely unlinked to the past—is a western phenomenon, some free will philosophy stuff. A quarter of my cultural background is quite fatalist in the outlook, so I see my decisions as the consequences of the laws of physics acting on the initial world state, and given same ‘random noise’, different decision by me implies both different future and different past.
Yep, that’s what the box being opaque means—the contents of the box have no causal effect on your perceptions.
Nope. Watching the video would contradict this principle as well, because you would still effectively be seeing the contents of the box.
What IS allowed by Newcomb’s problem, however, is coming to the conclusion that the contents of the box and your perceptions of Omega have a common cause in terms of how Omega functions or acts. You are then free to use that reasoning to work out what the contents of the box could be.
Your interpretation of Newcomb’s problem basically makes it incoherent. For example, let’s say I’m a CDT agent and I believe Omega predicted me correctly. Then, at the moment I make my decision to two-box, but before I actually see the contents of the opaque box, I already know that the opaque box is empty. Does this mean that the box is not “opaque”, by your reasoning?
If I don’t think Omega is able to predict me, then it’s not Newcomb’s problem, is it? Even if we assume that the Omega program is capable of predicting humans, DNA is not that likely to be sufficient evidence for it to be able to make good predictions.
Well, then it obviously depends on what that exact logic is.
First of all, as I said previously, if AIXI doesn’t want the money then the scenario is not Newcomb’s. Also, I don’t think the a_5 reading program will end up being the simplest explanation even in that scenario. The program would need to use something like a_5, a_67, a_166, a_190 and a_222 in each instance of Newcomb’s problem respectively. Rather than a world program with a generic “get inputs from AIXI” subroutine, you need a world program with a “recognize Newcomblike problems and use the appropriate bits” subroutine; there is still a complexity penalty.
Unless you’re trying to make a setup in which Omega necessarily works by magic, then given sufficient evidence of reality at large magic is always going to be penalised. Given that reality at large works in a non-magical way, explanations that bootstrap your model of reality at large are always going to be simpler than explanations that have to add extraneous elements of “magic” to the model.
Besides, if Omega is just plain magical, then Newcomb’s problem boils down to “is a million bigger than a thousand?”
Of course there can be predictors for AIXI. I can, for example, predict with a high degree of confidence that if AIXI knows what chess is and it wants to beat me at chess, it’s going to beat me. Also, if AIXI wants to maximise paperclips, I can easily predict that there are going to be a lot of paperclips.
By being the kind of person who one-boxes, I end up with a million dollars instead of a thousand.
Um, the “libertarian free will” perspective is mostly what I’m arguing against here. The whole problem with CDT is that it takes that perspective, and, in concluding that its action is not in any way caused by its past, it ends up with only $1000. My point is that AIXI ultimately suffers from the same problem; it assumes that it has this magical kind of free will when it actually does not, and also ends up with $1000.
Yeah, and then you kept stipulating that the model where Omega has read the action tape and then put or not put money into the box, but it didn’t leak onto sensory input, is very unlikely, and I noted that it’s stipulated in the problem statement that the box contents do not leak onto sensory input.
Let’s say AIXI lives inside the robot named Alice. According to every model employed by AIXI, the robot named Alice has pre-committed, since the beginning of time, to act out a specific sequence of actions. How the hell that assumes magical free will I don’t know. edit: and note that you can exclude machines which had read the action before printing matching sensory data, to actually ensure magical free will. I’m not even sure, maybe some variations by Hutter do just that.
edit:
That’s just abstruse. We both know what I mean.
Well, you’re just pre-committed to 1-box, then. The omegas that don’t know you’re pre-committed to 1-box (e.g. don’t trust you, can’t read your pre-committments, etc) would put nothing there, though, which you might be motivated to think about if its e.g. 10 millions vs 1 million, or 2 millions vs 1 million. (I wonder if one boxing is dependent on inflation...)
edit: let’s say I am playing the omega, and you know I know this weird trick for predicting you on the cheap.… you can work out what’s in the first box, can’t you? If you want money and don’t care of proving omega wrong out of spite, I can simply put nothing in the first box, and count on you to figure that out. You might have committed to the situation with 1 million vs 1 thousand, but I doubt you committed to 1000000 vs 999999 . You say you one box, fine, you get nothing—a rare time Omega is wrong.
edit2: a way to actually do Newcomb’s in real life, by the way. Take poor but not completely stupid people, make it 1000 000 vs 999 999 , and you can be almost always right. You can also draw some really rich people who you believe don’t really care and would 1-box for fun, and put a million in the first box for those, and be almost always right about both types of the case.
The two situations are quite different. Any complexity penalty for the non-leaking box has already been paid via AIXI’s observations of the box and the whole Newcomb’s setup; the opaqueness of the box just boils down to normal reality. On the other hand, your “action bit” model in which Omega reads AIXI’s action tape is associated with a significant complexity penalty because of the privileged nature of the situation—why specifically Omega, and not anyone else? Why does Omega specifically access that one bit, and not one of the other bits? The more physics-like and real AIXI’s Turing machines get, the more of a penalty will be associated with Turing machines that need to incorporate a special case for a specific event.
AIXI as defined by Hutter (not just some “variation”) has a foundational assumption that an action at time t cannot influence AIXI’s perceptions at times 1..t-1. This is entirely incompatible with a model of Alice where she has pre-commited since the beginning of time, because such an Alice would be able to discover her own pre-commitment before she took the action in question. AIXI, on the other hand, explicitly forbids world models where that can happen.
No, I don’t. My point is that although you can’t predict AIXI in the general case, there are still many cases where AIXI can be predicted with relative ease. My argument is still that Newcomb’s problem is one of those cases (and that AIXI two-boxes).
As for all of your scenarios with different Omegas or different amounts of money, obviously a major factor is how accurate I think Omega’s predictions are. If ze has only been wrong one time in a million, and this includes people who have been one-boxing as well, why should I spend much time thinking about the possibility that I could be the one time he gets it wrong?
Similarly, if you’re playing Omega and you don’t have a past history of correctly predicting one-boxing vs one-boxers, then yes, I two-box. However, that scenario isn’t Newcomb’s problem. For it to be Newcomb’s problem, Omega has to have a history of correctly predicting one-boxers as well as two-boxers.
I suspect the unspecified implementation of Omega hides assumptions if not contradictions. Let me propose a more concrete version: The problem is presented by Conservative Finite Omega (CFO), who works by pulling the agent’s source code, simulating it for a long but finite time, and putting $1,000,000 in the opaque box iff the simulation is determined to definitely one-box. The agent never walks away with the full $1,001,000, though the agent does sometimes walk away with $0.
So, assuming AIXI is confident in accurate models of how CFO works, CFO will simulate AIXI, which requires it to simulate AIXI’s (accurate) simulation of CFO—endless recursion. AIXI ‘wins’ the timeout war (correctly predicting CFO’s timeout), concludes that CFO has left the opaque box empty, and two-boxes.
You could look at that outcome as AIXI being penalized for being too smart. You could also say that an even smarter agent would prepend ‘if (facing CFO-like situation) then return one-box’ to its source code. Fundamentally, the specification of AIXI cannot conceive of its source code being an output; it’s baked into the assumptions that the explicit output bits are the only outputs.
Sure, I don’t necessarily blame the AIXI equation when it’s facing a relatively stupid Omega in that kind of situation.
However, consider “More Intelligent Finite Omega”, who pulls the agent’s source code and uses an approximate theorem-proving approach until it determines, with high confidence, what AIXI is going to do. Assuming that AIXI has received sufficient evidence to be reasonably confident in its model of MIFO, MIFO can reason like this:
AIXI will be able to accurately simulate me, therefore it will either have determined that the box is already empty, or already full.
Given either of those two models, AIXI will calculate that the best action is to two-box.
Consequently, AIXI will two-box.
and then MIFO will leave the opaque box empty, and its prediction will have been correct. Moreover, MIFO had no other choice; if it were put the money in the opaque box, AIXI would still have two-boxed, and MIFO’s prediction would have been incorrect.
If you’re allowed to make the assumption that AIXI is confident in its model of CFO and CFO knows this, then I can make the same assumption about MIFO.
I think you’re right. At first I was worried (here and previously in the thread) that the proof that AIXI would two-box was circular, but I think it works out if you fill in the language about terminating turing machines and stuff. I was going to write up my formalization, but once I went through it in my head your proof suddenly looked too obviously correct to be worth expanding.