1) The challenge is not solving this individual problem, but creating a general theory that happens to solve this special case automatically. Our current formalizations of UDT fail on ASP—they have no concept of “stop thinking”.
Okay.
So, the superintelligent UDT agent can essentially see through both boxes (whether it wants to or not… or, rather, has no concept of not wanting to). Sorry if this is a stupid question, but wouldn’t UDT one-box anyway, whether the box is empty or contains $1,000,000, for the same reason that it pays in Counterfactual Mugging and Parfit’s Hitchhiker? When the box is empty, it takes the empty box so that there will be possible worlds where the box is not empty (as it would pay the counterfactual mugger so that it will get $10,000 in the other half of worlds), and when the box is not empty, it takes only the one box (despite seeing the extra money in the other box) so that the world it’s in will weigh 50% rather than 0% (as it would pay the driver in Parfit’s Hitchhiker, despite it having “already happened”, so that the worlds in which the hitchhiker gives it a ride in the first place will weigh 100% rather than 0%).
In our current implementations of UDT, the agent won’t find any proof that one-boxing leads to the predictor predicting one-boxing, because the agent doesn’t “know” that it’s only going to use a small fraction of its computing resources while searching for the proof. Maybe a different implementation could fix that.
It’s not an implementation of UDT in the sense that it doesn’t talk about all possible programs and universal prior on them. If you consider UDT as generalizing to ADT, where probability assumptions are dropped, then sure.
Um, I don’t consider the universal prior to be part of UDT proper. UDT can run on top of any prior, e.g. when you use it to solve toy problems as Wei did, you use small specialized priors.
Okay.
So, the superintelligent UDT agent can essentially see through both boxes (whether it wants to or not… or, rather, has no concept of not wanting to). Sorry if this is a stupid question, but wouldn’t UDT one-box anyway, whether the box is empty or contains $1,000,000, for the same reason that it pays in Counterfactual Mugging and Parfit’s Hitchhiker? When the box is empty, it takes the empty box so that there will be possible worlds where the box is not empty (as it would pay the counterfactual mugger so that it will get $10,000 in the other half of worlds), and when the box is not empty, it takes only the one box (despite seeing the extra money in the other box) so that the world it’s in will weigh 50% rather than 0% (as it would pay the driver in Parfit’s Hitchhiker, despite it having “already happened”, so that the worlds in which the hitchhiker gives it a ride in the first place will weigh 100% rather than 0%).
In our current implementations of UDT, the agent won’t find any proof that one-boxing leads to the predictor predicting one-boxing, because the agent doesn’t “know” that it’s only going to use a small fraction of its computing resources while searching for the proof. Maybe a different implementation could fix that.
It’s not an implementation of UDT in the sense that it doesn’t talk about all possible programs and universal prior on them. If you consider UDT as generalizing to ADT, where probability assumptions are dropped, then sure.
Um, I don’t consider the universal prior to be part of UDT proper. UDT can run on top of any prior, e.g. when you use it to solve toy problems as Wei did, you use small specialized priors.
There are no priors used in those toy problems, just one utility definition of interest.
Well, the use of any priors over possible worlds is the thing I find objectionable.