I don’t understand how precisely an unlosing agent is supposed to work, but either it is equivalent to some expected utility maximizer, or it violates at least one of the VNM axioms. Which of these do you expect to be the case? If it is equivalent to some expected utility maximizer, then what do you get by formulating it as an unlosing agent instead? If it violates the VNM axioms, then isn’t that a good reason not to be an unlosing agent?
One thing that is clear is that if you faced that decision 3^^^^3 times, each decision independent from the others… then you should pay each time.
If you face that decision 3^^^^3 times and attempt to pay every time, you will almost instantly run out of money and find it physically impossible to pay every time. Unless you start out with $5*3^^^^3, in which case the marginal value of $5 to you is probably so trivially low that it makes sense to pay the mugger even in a one-shot Pascal’s mugger. I don’t see how the repitition changes anything.
… this would be an expected utility maximiser with a utility that’s unbounded....
There is literally no such thing as an expected utility maximiser with unbounded utility. It’s mathematically impossible. If your utility is unbounded, then there exist lotteries with infinite expected utility, and then you need some way of comparing series and picking a “bigger” one even though neither one of them converges. If you do that, that makes you something other than an expected utility maximiser, since the expected utility of such lotteries is simply “undefined”. This basically comes down to the fact from functional analysis that continuous linear maps on a Banach space are bounded. I have gone over this before, and am disappointed that people are still trying to defend them. It’s like saying that it is foolish to ride a horse because unicorns are more awesome, so you should ride one of those instead.
If there is more than one utility function that it could end up maximizing, then it is not an expected utility maximizer, because any particular utility function is better maximized by maximizing it directly than by possibly maximizing some other utility function depending on certain circumstances. As an example, suppose you could end up using one of two utility functions: u and v, there are three possible outcomes: X, Y, and Z, and u(X)>u(Y) while v(X)<v(Y). Consider two possible circumstances:
1) You get to choose between X and Y.
2) You get to choose between the lotteries .5X+.5Z and .5Y+.5Z.
If you would end up using u if (1) happens but end up using v if (2) happens, then you violate the independence axiom.
Relate this to value loading. If the programmer says cake, you value cake; if they say death, you value death. You could see this as choosing between two utilities, or you could see it as having a single utility function where “what the programmer says” strongly distinguishes between otherwise identical universes.
I don’t understand how precisely an unlosing agent is supposed to work, but either it is equivalent to some expected utility maximizer, or it violates at least one of the VNM axioms. Which of these do you expect to be the case? If it is equivalent to some expected utility maximizer, then what do you get by formulating it as an unlosing agent instead? If it violates the VNM axioms, then isn’t that a good reason not to be an unlosing agent?
If you face that decision 3^^^^3 times and attempt to pay every time, you will almost instantly run out of money and find it physically impossible to pay every time. Unless you start out with $5*3^^^^3, in which case the marginal value of $5 to you is probably so trivially low that it makes sense to pay the mugger even in a one-shot Pascal’s mugger. I don’t see how the repitition changes anything.
There is literally no such thing as an expected utility maximiser with unbounded utility. It’s mathematically impossible. If your utility is unbounded, then there exist lotteries with infinite expected utility, and then you need some way of comparing series and picking a “bigger” one even though neither one of them converges. If you do that, that makes you something other than an expected utility maximiser, since the expected utility of such lotteries is simply “undefined”. This basically comes down to the fact from functional analysis that continuous linear maps on a Banach space are bounded. I have gone over this before, and am disappointed that people are still trying to defend them. It’s like saying that it is foolish to ride a horse because unicorns are more awesome, so you should ride one of those instead.
It’s equivalent to an expected utility maximiser, but the utility function that it maximises is not determined ahead of time.
If there is more than one utility function that it could end up maximizing, then it is not an expected utility maximizer, because any particular utility function is better maximized by maximizing it directly than by possibly maximizing some other utility function depending on certain circumstances. As an example, suppose you could end up using one of two utility functions: u and v, there are three possible outcomes: X, Y, and Z, and u(X)>u(Y) while v(X)<v(Y). Consider two possible circumstances: 1) You get to choose between X and Y. 2) You get to choose between the lotteries .5X+.5Z and .5Y+.5Z.
If you would end up using u if (1) happens but end up using v if (2) happens, then you violate the independence axiom.
Here’s a better proof of the existence of unlosing agents: http://lesswrong.com/r/discussion/lw/knv/model_of_unlosing_agents/
Relate this to value loading. If the programmer says cake, you value cake; if they say death, you value death. You could see this as choosing between two utilities, or you could see it as having a single utility function where “what the programmer says” strongly distinguishes between otherwise identical universes.