I am confused about the iterated Parfit’s hitchhiker setup. In the one-shot PH agent not only never gets to safety, they die in the desert. So you must have something else in mind. Presumably they can survive the desert in some way (how?) at the cost of lower utility. Realizing that precommitting to not paying results in suboptimal outcomes, the agent would, well, “explore” other options, including precommitting to paying.
If my understanding is correct, then this game is isomorphic to betting on a coin toss:
You are told that you win $1000 if a coin lands heads, and $1 if the coin lands tails. What you do not know is that the coin is 100% biased and always lands tails.
In that less esoteric setup you will initially bet on tails, but after a few losses, realize that the coin is biased and adjust your bet accordingly.
In the iterated version we want the coin to sometimes be heads, sometimes tails. Sorry, I’m confused, I have no idea why you want to transform the problem like that?
But coin needs to depend on your prediction instead of being always biased a particular way.
Does it? I think it only depends on failure to explore/update which is a property of the (broken) agent, not an effect of the setup.
My recommended semi-causal agent (thanks to https://www.lesswrong.com/posts/9m2fzjNSJmd3yxxKG/acdt-a-hack-y-acausal-decision-theory for the idea) does the following: Start out with X% intend to heads/pay and 1-X intend to tails/not pay, based on priors of chance (NOT 0 or 1) and value of each payout box in the matrix. Randomize and commit before start of game (so the predictor can operate), adjust per bayes’ rule and re-randomize the committment after each iteration. You’ll never get stuck at certainty, so you’ll converge on the optimal percentage for the power of the predictor and the outcomes actually available.
This doesn’t work with computationally-infeasible ranges of action/outcome, but it seems to solve iterated (including iterated in thought-experiments) simple definable cases.
I am confused about the iterated Parfit’s hitchhiker setup. In the one-shot PH agent not only never gets to safety, they die in the desert. So you must have something else in mind. Presumably they can survive the desert in some way (how?) at the cost of lower utility. Realizing that precommitting to not paying results in suboptimal outcomes, the agent would, well, “explore” other options, including precommitting to paying.
If my understanding is correct, then this game is isomorphic to betting on a coin toss:
In that less esoteric setup you will initially bet on tails, but after a few losses, realize that the coin is biased and adjust your bet accordingly.
Yeah, need to replace dying with losing a lot of utility. I’ve updated the post.
But coin needs to depend on your prediction instead of being always biased a particular way.
I don’t see why, where would the isomorphism break?
In the iterated version we want the coin to sometimes be heads, sometimes tails. Sorry, I’m confused, I have no idea why you want to transform the problem like that?
Was trying to explain, but it looks like I screwed something up in the reformulation :)
Does it? I think it only depends on failure to explore/update which is a property of the (broken) agent, not an effect of the setup.
My recommended semi-causal agent (thanks to https://www.lesswrong.com/posts/9m2fzjNSJmd3yxxKG/acdt-a-hack-y-acausal-decision-theory for the idea) does the following: Start out with X% intend to heads/pay and 1-X intend to tails/not pay, based on priors of chance (NOT 0 or 1) and value of each payout box in the matrix. Randomize and commit before start of game (so the predictor can operate), adjust per bayes’ rule and re-randomize the committment after each iteration. You’ll never get stuck at certainty, so you’ll converge on the optimal percentage for the power of the predictor and the outcomes actually available.
This doesn’t work with computationally-infeasible ranges of action/outcome, but it seems to solve iterated (including iterated in thought-experiments) simple definable cases.