E(f) needn’t have anything to do with W. But it has to do with the f-values of all the different versions of you that there might be in the future (or in different Everett branches, or whatever). And it treats all of them symmetrically, looking only at their average and not at their distribution.
That is: the way in which you decide what to do can be expressed in terms of some preference you have about the state of the world, viewed from the perspective of one possible-future-you, but all those possible-future-yous have to be treated symmetrically, just averaging their preferences. (By “preference” I just mean anything that plays the role of f.)
So, Phil says, if that’s the only consistent way to treat all the different versions of you, then surely it’s also the only consistent way to treat all the different people in the world.
(This is of course the controversial bit. It’s far from obvious that you should see the possible-future-yous in the same way as you see the actual-other-people. For instance, because it’s more credible to think of those possible-future-yous as having the same utility function as one another. And because we all tend to care more about the welfare of our future selves than we do about other people’s. And so on.)
If so, then the following would be true: To act consistently, your actions must be such as to maximize the average of something (which, yes, need have nothing to do with the functions w, but it had better in some sense be the same something) over all actual and possible people.
I think Phil is wrong, but your criticisms don’t seem fair to me.
But it has to do with the f-values of all the different versions of you that there might be in the future (or in different Everett branches, or whatever).
I think this is part of the problem. Expected utility is based on epistemic uncertainty, which has nothing to do with objective frequency, including objective frequency across Everett branches or the like.
Nothing to do with objective frequency? Surely that’s wrong: e.g., as you gain information, your subjective probabilities ought to converge on the objective frequencies.
But I agree: the relevant sense of expectation here has to be the subjective one (since all an agent can use in deciding what to prefer is its own subjective probabilities, not whatever objective ones there may be), and this does seem like it’s a problem with what Phil’s saying.
the relevant sense of expectation here has to be the subjective one (since all an agent can use in deciding what to prefer is its own subjective probabilities, not whatever objective ones there may be), and this does seem like it’s a problem with what Phil’s saying.
It’s just as much a problem with all of decision theory, and all expectation maximization, as with anything I’m saying. This may be a difficulty, but it’s completely orthogonal to the issue at hand, since all of the alternatives have that same weakness.
I think the point is that, if probability is only in the mind, the analogy between averaging over future yous and averaging over other people is weaker than it might initially appear.
It sort of seems like there might be an analogy if you’re talking about averaging over versions of you that end up in different Everett branches. But if we’re talking about subjective probabilities, then there’s no sense in which these “future yous” exist, except in your mind, and it’s more difficult to see the analogy between averaging over them, and averaging over actual people.
Expected utility is based on epistemic uncertainty
What? So if you had accurate information about the probability distribution of outcomes, then you couldn’t use expected utility? I don’t think that’s right. In fact, it’s exactly the reverse. Expected utility doesn’t really work when you have epistemic uncertainty.
If I’m using subjective probabilities at all – if I don’t know the exact outcome – I’m working under uncertainty and using expected utility. If, in a multiverse, I know with certainty the objective frequency distribution of single-world outcomes, then yes, I just pick the action with the highest utility.
I don’t know that this affects your point, but I think we can make good sense of objective probabilities as being something else than either subjective probabilities or objective frequencies. See for example this.
If, in a multiverse, I know with certainty the objective frequency distribution of single-world outcomes, then yes, I just pick the action with the highest utility.
You still need to multiply utility of those events by their probability. Objective frequency is not as objective as it may seem, it’s just a point at which the posterior is no longer expected to change, given more information. Or, alternatively, it’s “physical probability”, a parameter in your model that has nothing to do with subjective probability and expected utility maximization, and has the status similar to that of, say, mass.
Expected utility is based on epistemic uncertainty, which has nothing to do with objective frequency
If you have accurate information about the probability distribution, you don’t have epistemic uncertainty.
What was you original comment supposed to mean?
If, in a multiverse, I know with certainty the objective frequency distribution of single-world outcomes, then yes, I just pick the action with the highest utility.
This is the situation which the theory of expected utility was designed for. I think your claim is exactly backwards of what it should be.
If you have accurate information about the probability distribution, you don’t have epistemic uncertainty.
This is not what is meant by epistemic uncertainty. In a framework of Bayesian probability theory, you start with a fixed, exactly defined prior distribution. The uncertainty comes from working with big events on state space, some of them coming in form of states of variables, as opposed to individual states. See Probability Space, Random Variable, also E.T. Jaynes (1990) “Probability Theory as Logic” may be helpful.
According to Wikipedia, that is what is meant by epistemic uncertainty. It says that one type of uncertainty is
“1. Uncertainty due to variability of input and / or model parameters when the characterization of the variability is available (e.g., with probability density functions, pdf),”
and that all other types of uncertainty are epistemic uncertainty.
And here’s a quote from “Separating natural and epistemic uncertainty in flood frequency analysis”, Bruno Merz and Annegret H. Thieken, J. of Hydrology 2004, which also agrees with me:
“Natural uncertainty stems from variability of the underlying stochastic process. Epistemic uncertainty results from incomplete knowledge about the process under study.”
This “natural uncertainty” is a property of distributions, while epistemic uncertainty to which you refer here corresponds to what I meant. When you have incomplete knowledge about the process under study, you are working with one of the multiple possible processes, you are operating inside a wide event that includes all these possibilities. I suspect you are still confusing the prior on global state space with marginal probability distributions on variables. Follow the links I gave before.
To act consistently, your actions must be such as to maximize the average of something
Yes, but maximizing the average of something implies neither utilitarianism nor indifference to equity, as Phil has claimed it does. I don’t see how pointing this out is unfair.
You understand what I’m saying, so I’d very much like to know why you think it’s wrong.
Note that I’m not claiming that average utilitarianism must be correct. The axioms could be unreasonable, or a strict proof could fail for some as-yet unknown reason. But I think the axioms are either reasonable in both cases, or unreasonable in both cases; and so expected-value maximization and average utilitarianism go together.
See (1) the paragraph in my comment above beginning “This is of course the controversial bit”, (2) Wei_Dai’s comment further down and my reply to it, and (3) Nick Tarleton’s (basically correct) objection to my description of E(f) as being derived from “the f-values of all the different versions of you”.
E(f) needn’t have anything to do with W. But it has to do with the f-values of all the different versions of you that there might be in the future.
I think this is part of where things are going wrong. f values aren’t things that each future version of me has. They are especially not the value that a particular future me places on a given outcome, or the preferences of that particular future me. f values are simply mathematical constructs built to formalize the choices that current me happens to make over gambles.
Despite its name, expected utility maximization is not actually averaging over the preferences of future me-s; it’s just averaging a more-or-less arbitrary function, that may or may not have anything to do with those preferences.
I think you may be missing PhilGoetz’s point.
E(f) needn’t have anything to do with W. But it has to do with the f-values of all the different versions of you that there might be in the future (or in different Everett branches, or whatever). And it treats all of them symmetrically, looking only at their average and not at their distribution.
That is: the way in which you decide what to do can be expressed in terms of some preference you have about the state of the world, viewed from the perspective of one possible-future-you, but all those possible-future-yous have to be treated symmetrically, just averaging their preferences. (By “preference” I just mean anything that plays the role of f.)
So, Phil says, if that’s the only consistent way to treat all the different versions of you, then surely it’s also the only consistent way to treat all the different people in the world.
(This is of course the controversial bit. It’s far from obvious that you should see the possible-future-yous in the same way as you see the actual-other-people. For instance, because it’s more credible to think of those possible-future-yous as having the same utility function as one another. And because we all tend to care more about the welfare of our future selves than we do about other people’s. And so on.)
If so, then the following would be true: To act consistently, your actions must be such as to maximize the average of something (which, yes, need have nothing to do with the functions w, but it had better in some sense be the same something) over all actual and possible people.
I think Phil is wrong, but your criticisms don’t seem fair to me.
I think this is part of the problem. Expected utility is based on epistemic uncertainty, which has nothing to do with objective frequency, including objective frequency across Everett branches or the like.
Nothing to do with objective frequency? Surely that’s wrong: e.g., as you gain information, your subjective probabilities ought to converge on the objective frequencies.
But I agree: the relevant sense of expectation here has to be the subjective one (since all an agent can use in deciding what to prefer is its own subjective probabilities, not whatever objective ones there may be), and this does seem like it’s a problem with what Phil’s saying.
It’s just as much a problem with all of decision theory, and all expectation maximization, as with anything I’m saying. This may be a difficulty, but it’s completely orthogonal to the issue at hand, since all of the alternatives have that same weakness.
I think the point is that, if probability is only in the mind, the analogy between averaging over future yous and averaging over other people is weaker than it might initially appear.
It sort of seems like there might be an analogy if you’re talking about averaging over versions of you that end up in different Everett branches. But if we’re talking about subjective probabilities, then there’s no sense in which these “future yous” exist, except in your mind, and it’s more difficult to see the analogy between averaging over them, and averaging over actual people.
What? So if you had accurate information about the probability distribution of outcomes, then you couldn’t use expected utility? I don’t think that’s right. In fact, it’s exactly the reverse. Expected utility doesn’t really work when you have epistemic uncertainty.
What does “accurate information about the probability distribution” mean? Probability is in the mind.
If I’m using subjective probabilities at all – if I don’t know the exact outcome – I’m working under uncertainty and using expected utility. If, in a multiverse, I know with certainty the objective frequency distribution of single-world outcomes, then yes, I just pick the action with the highest utility.
I don’t know that this affects your point, but I think we can make good sense of objective probabilities as being something else than either subjective probabilities or objective frequencies. See for example this.
You still need to multiply utility of those events by their probability. Objective frequency is not as objective as it may seem, it’s just a point at which the posterior is no longer expected to change, given more information. Or, alternatively, it’s “physical probability”, a parameter in your model that has nothing to do with subjective probability and expected utility maximization, and has the status similar to that of, say, mass.
You said
If you have accurate information about the probability distribution, you don’t have epistemic uncertainty.
What was you original comment supposed to mean?
This is the situation which the theory of expected utility was designed for. I think your claim is exactly backwards of what it should be.
This is not what is meant by epistemic uncertainty. In a framework of Bayesian probability theory, you start with a fixed, exactly defined prior distribution. The uncertainty comes from working with big events on state space, some of them coming in form of states of variables, as opposed to individual states. See Probability Space, Random Variable, also E.T. Jaynes (1990) “Probability Theory as Logic” may be helpful.
According to Wikipedia, that is what is meant by epistemic uncertainty. It says that one type of uncertainty is
“1. Uncertainty due to variability of input and / or model parameters when the characterization of the variability is available (e.g., with probability density functions, pdf),”
and that all other types of uncertainty are epistemic uncertainty.
And here’s a quote from “Separating natural and epistemic uncertainty in flood frequency analysis”, Bruno Merz and Annegret H. Thieken, J. of Hydrology 2004, which also agrees with me:
“Natural uncertainty stems from variability of the underlying stochastic process. Epistemic uncertainty results from incomplete knowledge about the process under study.”
This “natural uncertainty” is a property of distributions, while epistemic uncertainty to which you refer here corresponds to what I meant. When you have incomplete knowledge about the process under study, you are working with one of the multiple possible processes, you are operating inside a wide event that includes all these possibilities. I suspect you are still confusing the prior on global state space with marginal probability distributions on variables. Follow the links I gave before.
Yes, but maximizing the average of something implies neither utilitarianism nor indifference to equity, as Phil has claimed it does. I don’t see how pointing this out is unfair.
You understand what I’m saying, so I’d very much like to know why you think it’s wrong.
Note that I’m not claiming that average utilitarianism must be correct. The axioms could be unreasonable, or a strict proof could fail for some as-yet unknown reason. But I think the axioms are either reasonable in both cases, or unreasonable in both cases; and so expected-value maximization and average utilitarianism go together.
See (1) the paragraph in my comment above beginning “This is of course the controversial bit”, (2) Wei_Dai’s comment further down and my reply to it, and (3) Nick Tarleton’s (basically correct) objection to my description of E(f) as being derived from “the f-values of all the different versions of you”.
I think this is part of where things are going wrong. f values aren’t things that each future version of me has. They are especially not the value that a particular future me places on a given outcome, or the preferences of that particular future me. f values are simply mathematical constructs built to formalize the choices that current me happens to make over gambles.
Despite its name, expected utility maximization is not actually averaging over the preferences of future me-s; it’s just averaging a more-or-less arbitrary function, that may or may not have anything to do with those preferences.