This looks more like a problem with updating than with MMEU though. It seems possible to design a variant of UDT that uses MMEU, without it wanting to self-modify into something else (at least not for this reason).
I can’t see how this would work. Wouldn’t the UDT-ish approach be to ask an MMEU agent to pick a strategy once, before making any updates? The MMEU agent would choose a strategy that makes it equivalent to a Bayesian agent, as I describe. The characteristic ambiguity-averse behaviour only appears if the agent is allowed to update.
Given a Cartesian boundary between agent and environment, you could make an agent that prefers to have its future actions be those that are prescribed by MMEU, and you’d then get MMEU-like behaviour persisting upon reflection, but I assume this isn’t what you mean since it isn’t UDT-ish at all.
Suppose you program a UDT-MMEU agent to care about just one particular world defined by some world program. The world program takes a single bit as input, representing the mysterious coin, and the agent represents uncertainty about this bit using a probability interval. You think that in this world the agent will either be offered only bet 1, or only bet 2, or the world will split into two copies with the agent being offered a different bet in each copy (analogous to your example). You have logical uncertainty as to which is the case, but the UDT-MMEU agent can compute and find out for sure which is the case. (I’m assuming this agent isn’t updateless with regard to logical facts but just computes as many of them as it can before making decisions.) Then UDT-MMEU would reject the bet unless it turns out that the world does split in two.
Unless I made a mistake somewhere, it seems like UDT-MMEU does retain “ambiguity-averse behaviour” and isn’t equivalent to any standard UDT agent, except in the sense that if you did know which version of the bet would be offered in this world, you could design a UDT agent that does the same thing as the UDT-MMEU agent.
This looks more like a problem with updating than with MMEU though. It seems possible to design a variant of UDT that uses MMEU, without it wanting to self-modify into something else (at least not for this reason).
I can’t see how this would work. Wouldn’t the UDT-ish approach be to ask an MMEU agent to pick a strategy once, before making any updates? The MMEU agent would choose a strategy that makes it equivalent to a Bayesian agent, as I describe. The characteristic ambiguity-averse behaviour only appears if the agent is allowed to update.
Given a Cartesian boundary between agent and environment, you could make an agent that prefers to have its future actions be those that are prescribed by MMEU, and you’d then get MMEU-like behaviour persisting upon reflection, but I assume this isn’t what you mean since it isn’t UDT-ish at all.
Suppose you program a UDT-MMEU agent to care about just one particular world defined by some world program. The world program takes a single bit as input, representing the mysterious coin, and the agent represents uncertainty about this bit using a probability interval. You think that in this world the agent will either be offered only bet 1, or only bet 2, or the world will split into two copies with the agent being offered a different bet in each copy (analogous to your example). You have logical uncertainty as to which is the case, but the UDT-MMEU agent can compute and find out for sure which is the case. (I’m assuming this agent isn’t updateless with regard to logical facts but just computes as many of them as it can before making decisions.) Then UDT-MMEU would reject the bet unless it turns out that the world does split in two.
Unless I made a mistake somewhere, it seems like UDT-MMEU does retain “ambiguity-averse behaviour” and isn’t equivalent to any standard UDT agent, except in the sense that if you did know which version of the bet would be offered in this world, you could design a UDT agent that does the same thing as the UDT-MMEU agent.