Now, if we have a prior over the possible sets of lotteries you’ll be presented with, then for each decision procedure and each utility function, we have the expected utility given that you follow that decision procedure. These expected utilities give us a new sense of Pareto optimality: A non-optimizing decision procedure that is Pareto-optimal in your sense will not be Pareto-optimal with respect to these expected utilities.
Benja answered a similar point recently in this comment, in his third paragraph which starts with “I disagree”. If you apply the Pareto-optimal decision procedure to the prior instead of after updating, then it will be Pareto-optimal with respect to these expected utilities. And in general, given different priors the decision will be equivalent to maximizing different linear aggregations of individual utility functions, so you still have the same issue that the decision procedure as a function cannot be reproduced by EU maximization of a single linear aggregation.
You might ask, why does this matter if in real life we just have one prior to deal with? I guess the answer is that it’s a way of making clear that a Pareto-optimal decision procedure need not be algorithmically equivalent to EU maximization, so we can’t conclude that we should become EU maximizers instead of implementing some other algorithm, at least not based just on considerations of Pareto optimality.
ETA: Your response to this comment seems to indicate a misunderstanding. It’s probably easier to clear up this via online chat. I sent you a PM with my contact info.
Suppose you have a prior over all possible priors, and your first action after determining your utility function is to figure out which prior you should use. Before choosing a particular prior, you can define the expected utility of policies in terms of the “expected prior” of your distribution over priors. No matter how you arrived at your utility function, you will want to remember it as a linear combination of values while updating on the prior you chose.
So if I understand you correctly, if I wanted to switch from a non-optimizing policy to an optimizing policy, I’d have to choose whether to switch to a policy that’s Pareto-optimal with respect to my current beliefs, or to a policy that’s Pareto-optimal with respect to old beliefs. And if we don’t know which beliefs to use, we can hardly say that we “should” choose one or the other.
Benja answered a similar point recently in this comment, in his third paragraph which starts with “I disagree”. If you apply the Pareto-optimal decision procedure to the prior instead of after updating, then it will be Pareto-optimal with respect to these expected utilities. And in general, given different priors the decision will be equivalent to maximizing different linear aggregations of individual utility functions, so you still have the same issue that the decision procedure as a function cannot be reproduced by EU maximization of a single linear aggregation.
You might ask, why does this matter if in real life we just have one prior to deal with? I guess the answer is that it’s a way of making clear that a Pareto-optimal decision procedure need not be algorithmically equivalent to EU maximization, so we can’t conclude that we should become EU maximizers instead of implementing some other algorithm, at least not based just on considerations of Pareto optimality.
ETA: Your response to this comment seems to indicate a misunderstanding. It’s probably easier to clear up this via online chat. I sent you a PM with my contact info.
Suppose you have a prior over all possible priors, and your first action after determining your utility function is to figure out which prior you should use. Before choosing a particular prior, you can define the expected utility of policies in terms of the “expected prior” of your distribution over priors. No matter how you arrived at your utility function, you will want to remember it as a linear combination of values while updating on the prior you chose.
So if I understand you correctly, if I wanted to switch from a non-optimizing policy to an optimizing policy, I’d have to choose whether to switch to a policy that’s Pareto-optimal with respect to my current beliefs, or to a policy that’s Pareto-optimal with respect to old beliefs. And if we don’t know which beliefs to use, we can hardly say that we “should” choose one or the other.
Is that statement close to your point of view?