MMEU isn’t stable upon reflection. Suppose that in addition to the mysterious [0.4, 0.6] coin, you had a fair coin, and I tell you that all offer bet 1 (“pay 50¢ to be payed $1.10 if the coin came up heads”) if the fair coin comes up heads and bet 2 if the fair coin comes up tails, but you have to choose whether to accept or reject before flipping the fair coin to decide which bet will be chosen. In this case, the Knighian uncertainty cancels out, and your expected winnings are +5¢ no matter which value is [0.4, 0.6] is taken to be the true probabilty of the mysterious coin, so you would take this bet on MMEU.
Upon seeing how the fair coin turns out, however, MMEU would tell you to reject whichever of bets 1 and 2 is offered. Thus, if I offer to let you see the result of the fair coin before deciding whether to accept the bet, you will actually prefer not to see the coin, for an expected outcome of +5¢, rather than see the coin, reject the bet, and win nothing with certainty. Alternatively, if given the chance, you would prefer to self-modify so as to not exhibit ambiguity aversion in this scenario.
In general, any agent using a decision rule that is not generalized Bayesian performs strictly worse than some generalized Bayes decision rule. Note, though, that this does not mean that such an agent is forced to accept at least one of bets 1 and 2, since rejecting whichever of them is offered is a Bayes rule; for example, a Bayesian agent who believes that the bookie knows something that they don’t will behave in this way. It does mean, though, that there are many situations where MMEU cannot work, such as in my example above, since in such scenarios it is not equivalent to any Bayes rule.
In this case, the Knighian uncertainty cancels out
Does it? You still know that you will only be able to take one of the two bets; you just don’t know which one. The Knightian uncertainty only cancels out if you know you can take both bets.
This looks more like a problem with updating than with MMEU though. It seems possible to design a variant of UDT that uses MMEU, without it wanting to self-modify into something else (at least not for this reason).
I can’t see how this would work. Wouldn’t the UDT-ish approach be to ask an MMEU agent to pick a strategy once, before making any updates? The MMEU agent would choose a strategy that makes it equivalent to a Bayesian agent, as I describe. The characteristic ambiguity-averse behaviour only appears if the agent is allowed to update.
Given a Cartesian boundary between agent and environment, you could make an agent that prefers to have its future actions be those that are prescribed by MMEU, and you’d then get MMEU-like behaviour persisting upon reflection, but I assume this isn’t what you mean since it isn’t UDT-ish at all.
Suppose you program a UDT-MMEU agent to care about just one particular world defined by some world program. The world program takes a single bit as input, representing the mysterious coin, and the agent represents uncertainty about this bit using a probability interval. You think that in this world the agent will either be offered only bet 1, or only bet 2, or the world will split into two copies with the agent being offered a different bet in each copy (analogous to your example). You have logical uncertainty as to which is the case, but the UDT-MMEU agent can compute and find out for sure which is the case. (I’m assuming this agent isn’t updateless with regard to logical facts but just computes as many of them as it can before making decisions.) Then UDT-MMEU would reject the bet unless it turns out that the world does split in two.
Unless I made a mistake somewhere, it seems like UDT-MMEU does retain “ambiguity-averse behaviour” and isn’t equivalent to any standard UDT agent, except in the sense that if you did know which version of the bet would be offered in this world, you could design a UDT agent that does the same thing as the UDT-MMEU agent.
MMEU isn’t stable upon reflection. Suppose that in addition to the mysterious [0.4, 0.6] coin, you had a fair coin, and I tell you that all offer bet 1 (“pay 50¢ to be payed $1.10 if the coin came up heads”) if the fair coin comes up heads and bet 2 if the fair coin comes up tails, but you have to choose whether to accept or reject before flipping the fair coin to decide which bet will be chosen. In this case, the Knighian uncertainty cancels out, and your expected winnings are +5¢ no matter which value is [0.4, 0.6] is taken to be the true probabilty of the mysterious coin, so you would take this bet on MMEU.
Upon seeing how the fair coin turns out, however, MMEU would tell you to reject whichever of bets 1 and 2 is offered. Thus, if I offer to let you see the result of the fair coin before deciding whether to accept the bet, you will actually prefer not to see the coin, for an expected outcome of +5¢, rather than see the coin, reject the bet, and win nothing with certainty. Alternatively, if given the chance, you would prefer to self-modify so as to not exhibit ambiguity aversion in this scenario.
In general, any agent using a decision rule that is not generalized Bayesian performs strictly worse than some generalized Bayes decision rule. Note, though, that this does not mean that such an agent is forced to accept at least one of bets 1 and 2, since rejecting whichever of them is offered is a Bayes rule; for example, a Bayesian agent who believes that the bookie knows something that they don’t will behave in this way. It does mean, though, that there are many situations where MMEU cannot work, such as in my example above, since in such scenarios it is not equivalent to any Bayes rule.
Does it? You still know that you will only be able to take one of the two bets; you just don’t know which one. The Knightian uncertainty only cancels out if you know you can take both bets.
This looks more like a problem with updating than with MMEU though. It seems possible to design a variant of UDT that uses MMEU, without it wanting to self-modify into something else (at least not for this reason).
I can’t see how this would work. Wouldn’t the UDT-ish approach be to ask an MMEU agent to pick a strategy once, before making any updates? The MMEU agent would choose a strategy that makes it equivalent to a Bayesian agent, as I describe. The characteristic ambiguity-averse behaviour only appears if the agent is allowed to update.
Given a Cartesian boundary between agent and environment, you could make an agent that prefers to have its future actions be those that are prescribed by MMEU, and you’d then get MMEU-like behaviour persisting upon reflection, but I assume this isn’t what you mean since it isn’t UDT-ish at all.
Suppose you program a UDT-MMEU agent to care about just one particular world defined by some world program. The world program takes a single bit as input, representing the mysterious coin, and the agent represents uncertainty about this bit using a probability interval. You think that in this world the agent will either be offered only bet 1, or only bet 2, or the world will split into two copies with the agent being offered a different bet in each copy (analogous to your example). You have logical uncertainty as to which is the case, but the UDT-MMEU agent can compute and find out for sure which is the case. (I’m assuming this agent isn’t updateless with regard to logical facts but just computes as many of them as it can before making decisions.) Then UDT-MMEU would reject the bet unless it turns out that the world does split in two.
Unless I made a mistake somewhere, it seems like UDT-MMEU does retain “ambiguity-averse behaviour” and isn’t equivalent to any standard UDT agent, except in the sense that if you did know which version of the bet would be offered in this world, you could design a UDT agent that does the same thing as the UDT-MMEU agent.