In my opinion, LWers should not give expected utility maximization the same axiomatic status that they award consequentialism. Is this worth a top level post?
There is a model which is standard in economics which say “people maximize expected utility; risk averseness arises because utility functions are concave”. This has always struck me as extremely fishy, for two reasons: (a) it gives rise to paradoxes like this, and (b) it doesn’t at all match what making a choice feels like for me: if someone offers me a risky bet, I feel inclined to reject it because it is risky, not because I have done some extensive integration of my utility function over all possible outcomes. So it seems a much safer assumption to just assume that people’s preferences are a function from probability distributions of outcomes, rather than making the more restrictive assumption that that function has to arise as an integral over utilities of individual outcomes.
So why is the “expected utility” model so popular? A couple of months ago I came across a blog-post which provides one clue: it pointed out that standard zero-sum game theory works when players maximize expected utility, but does not work if they have preferences about probability distributions of outcomes (since then introducing mixed strategies won’t work).
So an economist who wants to apply game theory will be inclined to assume that actors are maximizing expected utility; but we LWers shouldn’t necessarily.
There is a model which is standard in economics which say “people maximize expected utility; risk averseness arises because utility functions are convex”.
Do you mean concave?
A couple of months ago I came across a blog-post which provides one clue: it pointed out that standard zero-sum game theory works when players maximize expected utility, but does not work if they have preferences about probability distributions of outcomes (since then introducing mixed strategies won’t work).
Technically speaking, isn’t maximizing expected utility a special case of having preferences about probability distributions about outcomes? So maybe you should instead say “does not work elegantly if they have arbitrary preferences about probability distributions.”
This is what I tend to do when I’m having conversations in real life; let’s see how it works online :-)
I think I and John Maxwell IV mean the same thing, but here is the way I would phrase it. Suppose someone is offering me the pick a ticket for one of a range of different lotteries. Each lottery offers the same set of prizes, but depending on which lottery I participate in, the probability of winning them is different.
I am an agent, and we assume I have a preference order on the lotteries—e.g. which ticket I want the most, which ticket I want the least, and which tickets I am indifferent between. The action that will be rational for me to take depends on which ticket I want.
I am saying that a general theory of rational action should deal with arbitrary preference orders for the tickets. The more standard theory restricts attention to preference orders that arise from first assigning a utility value to each prize and then computing the expected utility for each ticket.
Let’s define an “experiment” as something that randomly changes an agent’s utility based on some probability density function. An agent’s “desire” for a given experiment is the amount of utility Y such that the agent is indifferent between the experiment occurring and having their utility changed by Y.
From Pfft we see that economists assume that for any given agent and any given experiment, the agent’s desire for the experiment is equal to
dx), where x is an amount of utility and f(x) gives the probability that the experiment’s outcome will be changing the agent’s utility by x. In other words, economists assume that agents desire experiments according to their expectation, which is not necessarily a good assumption.
Hmm… I hope you interpret your own words so that what you write comes out correct, your language is imprecise and at first I didn’t see a way to read what you wrote that made sense.
When I reread your comment to which I asked my question with this new perspective, the question disappeared. By “preference about probability distributions” you simply mean preference over events, that doesn’t necessarily satisfy expected utility axioms.
ETA: Note that in this case, there isn’t necessarily a way of assigning (subjective) probabilities, as subjective probabilities follow from preferences, but only if the preferences are of the right form. Thus, saying that those not-expected-utility preferences are over probability distributions is more conceptually problematic than saying that they are over events. If you don’t use probabilities in the decision algorithm, probabilities don’t mean anything.
Hmm… I hope you interpret your own words so that what you write comes out correct, your language is imprecise and at first I didn’t see a way to read what you wrote that made sense.
I am eager to improve. Please give specific suggestions.
By “preference about probability distributions” you simply mean preference over events, that doesn’t necessarily satisfy expected utility axioms.
Right.
Note that in this case, there isn’t necessarily a way of assigning (subjective) probabilities, as subjective probabilities follow from preferences, but only if the preferences are of the right form.
Hm? I thought subjective probabilities followed from prior probabilities and observed evidence and stuff. What do preferences have to do with them?
Thus, saying that those not-expected-utility preferences are over probability distributions is more conceptually problematic than saying that they are over events.
Are you using my technical definition of event or the standard definition?
Probably I should not have redefined “event”; I now see that my use is nonstandard. Hopefully I can clarify things. Let’s say I am going to roll a die and give you a number of dollars equal to the number of spots on the face left pointing upward. According to my (poorly chosen) use of the word “event”, the process of rolling the die is an “event”. According to what I suspect the standard definition is, the die landing with 4 spots face up would be an “event”. To clear things up, I suggest that we refer to the rolling of the die as an “experiment”, and 4 spots landing face up as an “outcome”. I’m going to rewrite my comment with this new terminology. I’m also replacing “value” with “desire”, for what it’s worth.
If you don’t use probabilities in the decision algorithm, probabilities don’t mean anything.
The way I want to evaluate the desirability of an experiment is more complicated than simply computing its expected value. But I still use probabilities. I would not give Pascal’s mugger any money. I would think very carefully about an experiment that had a 99% probability of getting me killed and a 1% probability of generating 101 times as much utility as I expect to generate in my lifetime, whereas a perfect expected utility maximizer would take this deal in an instant. Etc.
Roughly speaking, event is a set of alternative possibilities. So, the whole roll of a die is an event (set of all possible outcomes of a roll), as well as specific outcomes (sets that contain a single outcome). See probability space for a more detailed definition.
One way of defining prior and utility is just by first taking a preference over the events of sample space, and then choosing any pair prior+utility such that expected utility calculated from them induces the same order on events. Of course, the original order on events has to be “nice” in some sense for it to be possible to find prior+utility that have this property.
Any observations and updating consist in choosing what events you work with. Once prior is fixed, it never changes.
(Of course, you should read up on the subject in greater detail than I hint at.)
One way of defining prior and utility is just by first taking a preference over the events of sample space, and then choosing any pair prior+utility such that expected utility calculated from them induces the same order on events.
Um, isn’t that obviously wrong? It sounds like your are suggesting that we say “I like playing blackjack better than playing the lottery, so I should choose a prior probability of winning each and a utility associated with winning each so that that preference will remain consistent when I switch from ‘preference mode’ to ‘utilitarian mode’.” Wouldn’t it be better to choose the utilities of winning based on the prizes they give? And choose the priors for each based on studying the history of each game carefully?
Any observations and updating consist in choosing what events you work with. Once prior is fixed, it never changes.
Events are sets of outcomes, right? It sounds like you are suggesting that people update their probabilities by reshuffling which outcomes go with which events. Aren’t events just a layer of formality over outcomes? Isn’t real learning what happens when you change your estimations of the probabilities of outcomes, not when you reclassify them?
It almost seems to me as if we are talking past each other… I think I need a better background on this stuff. Can you recommend any books that explain probability for the layman? I already read a large section of one, but apparently it wasn’t very good...
Although I do think there is a chance you are wrong. I see you mixing up outcome-desirability estimates with chance-of-outcome estimates, which seems obviously bad.
If you don’t want the choice of preference to turn out bad for you, choose good preference ;-) There is no freedom in choosing your preference, as the “choice” is itself a decision-concept, defined in terms of preference, and can’t be a party to the definition of preference. When you are speaking of a particular choice of preference being bad or foolish, you are judging this choice from the reference frame of some other preference, while with preference as foundation of decision-making, you can’t go through this step. It really is that arbitrary. See also: Priors as Mathematical Objects, Probability is Subjectively Objective.
You are confusing probability space and its prior (the fundamental structure that bind the rest together) with random variables and their probability distributions (things that are based on probability space and that “interact” with each other through the definition in terms of the common probability space, restricted to common events). Informally, when you update a random variable given evidence (event) X, it means that you recalculate the probability distribution of that variable only based on the remaining elements of the probability space within event X. Since this can often be done using other probability distributions of various variables lying around, you don’t always see the probability space explicitly.
Why is rejection of mathematical expectation an unworkable solution?
This isn’t the only scenario where straight expectation is problematic. Pascal’s Mugging, timeless decision theory, and maximization of expected growth rate come to mind. That makes four.
In my opinion, LWers should not give expected utility maximization the same axiomatic status that they award consequentialism. Is this worth a top level post?
This is exactly my take on it also.
There is a model which is standard in economics which say “people maximize expected utility; risk averseness arises because utility functions are concave”. This has always struck me as extremely fishy, for two reasons: (a) it gives rise to paradoxes like this, and (b) it doesn’t at all match what making a choice feels like for me: if someone offers me a risky bet, I feel inclined to reject it because it is risky, not because I have done some extensive integration of my utility function over all possible outcomes. So it seems a much safer assumption to just assume that people’s preferences are a function from probability distributions of outcomes, rather than making the more restrictive assumption that that function has to arise as an integral over utilities of individual outcomes.
So why is the “expected utility” model so popular? A couple of months ago I came across a blog-post which provides one clue: it pointed out that standard zero-sum game theory works when players maximize expected utility, but does not work if they have preferences about probability distributions of outcomes (since then introducing mixed strategies won’t work).
So an economist who wants to apply game theory will be inclined to assume that actors are maximizing expected utility; but we LWers shouldn’t necessarily.
Do you mean concave?
Technically speaking, isn’t maximizing expected utility a special case of having preferences about probability distributions about outcomes? So maybe you should instead say “does not work elegantly if they have arbitrary preferences about probability distributions.”
This is what I tend to do when I’m having conversations in real life; let’s see how it works online :-)
Yes, thanks. I’ve fixed it.
What does it mean, technically, to have a preference “about” probability distributions?
I think I and John Maxwell IV mean the same thing, but here is the way I would phrase it. Suppose someone is offering me the pick a ticket for one of a range of different lotteries. Each lottery offers the same set of prizes, but depending on which lottery I participate in, the probability of winning them is different.
I am an agent, and we assume I have a preference order on the lotteries—e.g. which ticket I want the most, which ticket I want the least, and which tickets I am indifferent between. The action that will be rational for me to take depends on which ticket I want.
I am saying that a general theory of rational action should deal with arbitrary preference orders for the tickets. The more standard theory restricts attention to preference orders that arise from first assigning a utility value to each prize and then computing the expected utility for each ticket.
Let’s define an “experiment” as something that randomly changes an agent’s utility based on some probability density function. An agent’s “desire” for a given experiment is the amount of utility Y such that the agent is indifferent between the experiment occurring and having their utility changed by Y.
From Pfft we see that economists assume that for any given agent and any given experiment, the agent’s desire for the experiment is equal to
Hmm… I hope you interpret your own words so that what you write comes out correct, your language is imprecise and at first I didn’t see a way to read what you wrote that made sense.
When I reread your comment to which I asked my question with this new perspective, the question disappeared. By “preference about probability distributions” you simply mean preference over events, that doesn’t necessarily satisfy expected utility axioms.
ETA: Note that in this case, there isn’t necessarily a way of assigning (subjective) probabilities, as subjective probabilities follow from preferences, but only if the preferences are of the right form. Thus, saying that those not-expected-utility preferences are over probability distributions is more conceptually problematic than saying that they are over events. If you don’t use probabilities in the decision algorithm, probabilities don’t mean anything.
I am eager to improve. Please give specific suggestions.
Right.
Hm? I thought subjective probabilities followed from prior probabilities and observed evidence and stuff. What do preferences have to do with them?
Are you using my technical definition of event or the standard definition?
Probably I should not have redefined “event”; I now see that my use is nonstandard. Hopefully I can clarify things. Let’s say I am going to roll a die and give you a number of dollars equal to the number of spots on the face left pointing upward. According to my (poorly chosen) use of the word “event”, the process of rolling the die is an “event”. According to what I suspect the standard definition is, the die landing with 4 spots face up would be an “event”. To clear things up, I suggest that we refer to the rolling of the die as an “experiment”, and 4 spots landing face up as an “outcome”. I’m going to rewrite my comment with this new terminology. I’m also replacing “value” with “desire”, for what it’s worth.
The way I want to evaluate the desirability of an experiment is more complicated than simply computing its expected value. But I still use probabilities. I would not give Pascal’s mugger any money. I would think very carefully about an experiment that had a 99% probability of getting me killed and a 1% probability of generating 101 times as much utility as I expect to generate in my lifetime, whereas a perfect expected utility maximizer would take this deal in an instant. Etc.
Roughly speaking, event is a set of alternative possibilities. So, the whole roll of a die is an event (set of all possible outcomes of a roll), as well as specific outcomes (sets that contain a single outcome). See probability space for a more detailed definition.
One way of defining prior and utility is just by first taking a preference over the events of sample space, and then choosing any pair prior+utility such that expected utility calculated from them induces the same order on events. Of course, the original order on events has to be “nice” in some sense for it to be possible to find prior+utility that have this property.
Any observations and updating consist in choosing what events you work with. Once prior is fixed, it never changes.
(Of course, you should read up on the subject in greater detail than I hint at.)
Um, isn’t that obviously wrong? It sounds like your are suggesting that we say “I like playing blackjack better than playing the lottery, so I should choose a prior probability of winning each and a utility associated with winning each so that that preference will remain consistent when I switch from ‘preference mode’ to ‘utilitarian mode’.” Wouldn’t it be better to choose the utilities of winning based on the prizes they give? And choose the priors for each based on studying the history of each game carefully?
Events are sets of outcomes, right? It sounds like you are suggesting that people update their probabilities by reshuffling which outcomes go with which events. Aren’t events just a layer of formality over outcomes? Isn’t real learning what happens when you change your estimations of the probabilities of outcomes, not when you reclassify them?
It almost seems to me as if we are talking past each other… I think I need a better background on this stuff. Can you recommend any books that explain probability for the layman? I already read a large section of one, but apparently it wasn’t very good...
Although I do think there is a chance you are wrong. I see you mixing up outcome-desirability estimates with chance-of-outcome estimates, which seems obviously bad.
If you don’t want the choice of preference to turn out bad for you, choose good preference ;-) There is no freedom in choosing your preference, as the “choice” is itself a decision-concept, defined in terms of preference, and can’t be a party to the definition of preference. When you are speaking of a particular choice of preference being bad or foolish, you are judging this choice from the reference frame of some other preference, while with preference as foundation of decision-making, you can’t go through this step. It really is that arbitrary. See also: Priors as Mathematical Objects, Probability is Subjectively Objective.
You are confusing probability space and its prior (the fundamental structure that bind the rest together) with random variables and their probability distributions (things that are based on probability space and that “interact” with each other through the definition in terms of the common probability space, restricted to common events). Informally, when you update a random variable given evidence (event) X, it means that you recalculate the probability distribution of that variable only based on the remaining elements of the probability space within event X. Since this can often be done using other probability distributions of various variables lying around, you don’t always see the probability space explicitly.
Well, rejection’s not a solution per se until you pick something justifiable to replace it with.
I’d be interested in a top-level post on the subject.