Prediction from reading this post before I delved into the paper: the controversy is going to be about psychology, not decision theory. (After delving into the paper: I’m going to go with ‘prediction confirmed.’)
So, he uses six axioms. How do they map onto Howard’s 5 that I used? It looks like his 0 is essentially “you can state the problem,” his 1 and 2 are my choice, his 3 and 4 don’t seem to have mirrors, and his 5 is my equivalence. I find it a little worrisome that three of my axioms don’t appear to show up in his- probability, order, and substitution- except possibly in 0. They’re clearly present in his analysis, but they feel like things that should be axioms instead of just taken for granted, and it’s not clear to me why he needed to raise 3 and 4 to the level of axioms.
It’s also not clear to me why he puts such emphasis on the independence and “sure-thing” principle. The “sure-thing” principle is widely held to only apply to a certain class of utility functions / “sure things”, and there’s not a good reason to expect people should or do have those utility functions. (Outcomes, properly understood, are the entire future- and so a game in which I flip a coin and you win or lose $100 can be different from a game in which I flip a coin and give you either $0 or $200, because you’re $100 richer in the second game.) Similarly, independence only holds for normatively correct probability functions, and so if you allow normatively incorrect probability functions you have to throw out independence.
(The difference between independence and “sure-thing” is that “sure-thing” applies a transformation to all of the outcomes, which may not be the same transformation to all of the utilities of those outcomes, and independence applies a transformation to all of the probabilities, which will be the same transformation to all of the act utilities for normatively correct probability functions.)
For the controversy, I’ll quote him directly:
I am aware of two objections to the proposed ex ante axiomatization. The point of departure of the first objection is the reasonable suspicion that people simply do not have numerical utilities in their heads, or at least have no access to them. Therefore, to assume that utility is an extensive structure is unreasonable since it seems practically impossible for decision makers to state the preferences required for obtaining the utility function. The same holds true for the probability function; where does the exogenously defined probability function come from?
The probability complaint is uninteresting. The “how do we measure utilities?” complaint is serious but a little involved to discuss.
Basically, there are three branches of rationality: epistemic, instrumental, and terminal (I’m not sure I like “terminal rationality” as a name; please suggest alternatives): thinking about uncertainties and probabilities, thinking about actions, and thinking about outcomes.
Peterson’s root complaint is that traditional decision theory is silent on the valuable parts of terminal rationality- it only ensures that your elicited preferences are consistent and then uses them. If they’re consistent but insane, the expected utility maximization won’t throw up any error flags (for example, Danzig’s diet optimization which prescribed 500 gallons of vinegar a day), because checking for sanity is not its job.
But pointing that complaint at decision theory seems mistaken, because it’s a question of how you build the utility function. The traditional approach he describes in lukeprog’s post above (the woman considering divorce) uses casuistry, expecting that people can elicit their preferences well about individual cases and then extrapolate. I think the approach he prefers is deconstruction- isolate the different desires that are relevant to the outcomes in question, construct (potentially nonlinear) tradeoffs between them, then order the outcomes, then figure out the optimal action. The first checks primarily for internal consistency; the second checks primarily for reflective equilibrium. But both can do each, and they can be used as complements instead of substitutes.
TL;DR: There does not appear to be meat to the controversy over axioms, and if there is then Peterson’s axioms strike me as worse than Howard’s and possibly worse than vNM or Savage. There is meat to the controversy over discovering utility functions, but I don’t think the 2004 paper you link is a valuable addition to that controversy. Compare it to the chapter on choice under certainty from Thinking and Deciding.
I’m confused by several parts of your reply. I’ll select just two of them for discussion. Perhaps you’ll have the motivation to try to un-confuse me.
I don’t understand your points about the axioms he uses. He uses the axioms required to derive the useful results he aimed for, given his approach to formalizing decision problems, and no more than that. Do you reject the plausibility of one of his axioms, or do you disagree that his results follow from his axioms?
I don’t think Peterson denies the usefulness of traditional axiomatic decision theory for checking the consistency of one’s preferences, he’s just saying that it would also be nice to have a decision theory that can tell you what you should choose given what you believe and what you value. Indeed, this is what many/most people actually do when trying to make decisions “rationally,” but this norm wasn’t justified with an axiomatic approach until Peterson (as far as I can tell).
He uses the axioms required to derive the useful results he aimed for, given his approach to formalizing decision problems, and no more than that.
Can you give me two examples of useful results he derives from the axioms? That’ll help me target my response. (I should note that the commentary in the grandparent is targeted at the 2004 paper in the context of the other things you’ve quoted on this page; if there’s relevant material in one of the other links I probably missed it.)
I don’t think Peterson denies the usefulness of traditional axiomatic decision theory for checking the consistency of one’s preferences, he’s just saying that it would also be nice to have a decision theory that can tell you what you should choose given what you believe and what you value.
Agreed. In this comment I want to differentiate between “decision theory” and a component of it, “expected utility theory” (I didn’t differentiate between them in the grandparent). The first studies how to make decisions, and the second studies a particular mathematical technique to isolate the highest scoring of a set of alternative actions. My claim is that expected utility theory is and should be silent on the design of human-appropriate utility functions, but that decision theory should include a component focused on the design of human-appropriate utility functions. That component will be primarily researched by psychologists- what makes humans happy, what do humans want, how do we align those, what common mistakes do humans make, what intuitions do humans have and when are those useful, and so on.
Peterson’s axioms look to me like trying to shoehorn human-appropriate utility functions into expected utility theory, which doesn’t seem to augment the math of calculating expected utilities or augment the actual design of human-appropriate utility functions. As far as I can tell, that field is too young to profit from an axiomatic approach.
But I said “profit” from axioms and you said “justified” with axioms, and those are different things. It’s not clear to me that Peterson’s axioms are useful at justifying the use of expected utility theory, and my hesitance hinges on the phrase “given what you believe and what you value” from the parent. That means that’s Peterson’s decision theory takes your beliefs and values as inputs and outputs decisions- which is exactly what traditional decision theory does, and so they look the same to me (and if they’re different, I think it’s because Peterson made his worse, not better). The underlying problem as I see it is that beliefs and values are not given, they have to be extracted- and traditional decision theory underestimated the difficulty of that extraction.
(Side note: decision theory underestimating the difficulty and decision theorists underestimating the difficulty are very different things. Indeed, it’s likely that decision theorists realized the problem was very hard, and so left it to the reader so they wouldn’t have to do it!)
Then the question is how much Peterson 2004 helps its readers extract their beliefs and values. As far as I can tell, there’s very little normative or prescriptive content.
My claim is that expected utility theory is and should be silent on the design of human-appropriate utility functions, but that decision theory should include a component focused on the design of human-appropriate utility functions.
What do you mean by “the design of human-appropriate utility functions”?
Can you give me two examples of useful results he derives from the axioms? That’ll help me target my response.
Actually, let me show you a section of Peterson (2009), which is an updated and (I think) clearer presentation of his axiomatic ex ante approach. It is a bit informal, but is mercifully succinct. (The longer, formal presentation is in Peterson 2008). Here is a PDF I made of the relevant section of Peterson (2009). It’s a bit blurry, but it’s readable.
What do you mean by “the design of human-appropriate utility functions”?
A utility function that accurately reflects the beliefs and values of the human it’s designed for. Someone looking for guidance would get assistance in discovering what their beliefs and values about the situation are, rather than just math help and a consistency check. Similarly, someone could accidentally write a utility function that drowns them in vinegar, and it would be nice if the decision-making apparatus noticed and didn’t.
That’s my interpretation of “he’s just saying that it would also be nice to have a decision theory that can tell you what you should choose given what you believe and what you value.”
Actually, let me show you a section of Peterson (2009), which is an updated and (I think) clearer presentation of his axiomatic ex ante approach.
This looks like it boils down to “the utility of an act is the weighted sum of the utility of its consequences.” It’s not clear to me what good formulating it like that does, and I don’t like that axiom 4 from the 2009 version looks circular. (You’re allowed to adjust the utility of different equiprobable outcomes so long as the total utility of the act is preserved. But, uh, aren’t we trying to prove that we can calculate the utility of an act with multiple possible outcome utilities, and haven’t we only assumed that it works for acts with only one possible outcome utility?)
“outcome”; “ultimate” (or “ultima” if you prefer Latin); “intrinsic” (to draw a contrast with “instrumental”); “telikos” (transliteration of the Greek for final, ultimate, terminal, or last); “endpoint”; “consequence”; “effect”; “impact”; “goal.”
I’m not sure about “value” in this context. The term “value” could attach to either acts or outcomes, I think. So, if the goal is to distinguish rationality that cares about acts first from rationality that cares about outcomes first, then “value” doesn’t seem to do a very good job. Does that sound right to you, or am I missing something about the distinction that you want to draw?
Prediction from reading this post before I delved into the paper: the controversy is going to be about psychology, not decision theory. (After delving into the paper: I’m going to go with ‘prediction confirmed.’)
So, he uses six axioms. How do they map onto Howard’s 5 that I used? It looks like his 0 is essentially “you can state the problem,” his 1 and 2 are my choice, his 3 and 4 don’t seem to have mirrors, and his 5 is my equivalence. I find it a little worrisome that three of my axioms don’t appear to show up in his- probability, order, and substitution- except possibly in 0. They’re clearly present in his analysis, but they feel like things that should be axioms instead of just taken for granted, and it’s not clear to me why he needed to raise 3 and 4 to the level of axioms.
It’s also not clear to me why he puts such emphasis on the independence and “sure-thing” principle. The “sure-thing” principle is widely held to only apply to a certain class of utility functions / “sure things”, and there’s not a good reason to expect people should or do have those utility functions. (Outcomes, properly understood, are the entire future- and so a game in which I flip a coin and you win or lose $100 can be different from a game in which I flip a coin and give you either $0 or $200, because you’re $100 richer in the second game.) Similarly, independence only holds for normatively correct probability functions, and so if you allow normatively incorrect probability functions you have to throw out independence.
(The difference between independence and “sure-thing” is that “sure-thing” applies a transformation to all of the outcomes, which may not be the same transformation to all of the utilities of those outcomes, and independence applies a transformation to all of the probabilities, which will be the same transformation to all of the act utilities for normatively correct probability functions.)
For the controversy, I’ll quote him directly:
The probability complaint is uninteresting. The “how do we measure utilities?” complaint is serious but a little involved to discuss.
Basically, there are three branches of rationality: epistemic, instrumental, and terminal (I’m not sure I like “terminal rationality” as a name; please suggest alternatives): thinking about uncertainties and probabilities, thinking about actions, and thinking about outcomes.
Peterson’s root complaint is that traditional decision theory is silent on the valuable parts of terminal rationality- it only ensures that your elicited preferences are consistent and then uses them. If they’re consistent but insane, the expected utility maximization won’t throw up any error flags (for example, Danzig’s diet optimization which prescribed 500 gallons of vinegar a day), because checking for sanity is not its job.
But pointing that complaint at decision theory seems mistaken, because it’s a question of how you build the utility function. The traditional approach he describes in lukeprog’s post above (the woman considering divorce) uses casuistry, expecting that people can elicit their preferences well about individual cases and then extrapolate. I think the approach he prefers is deconstruction- isolate the different desires that are relevant to the outcomes in question, construct (potentially nonlinear) tradeoffs between them, then order the outcomes, then figure out the optimal action. The first checks primarily for internal consistency; the second checks primarily for reflective equilibrium. But both can do each, and they can be used as complements instead of substitutes.
TL;DR: There does not appear to be meat to the controversy over axioms, and if there is then Peterson’s axioms strike me as worse than Howard’s and possibly worse than vNM or Savage. There is meat to the controversy over discovering utility functions, but I don’t think the 2004 paper you link is a valuable addition to that controversy. Compare it to the chapter on choice under certainty from Thinking and Deciding.
I’m confused by several parts of your reply. I’ll select just two of them for discussion. Perhaps you’ll have the motivation to try to un-confuse me.
I don’t understand your points about the axioms he uses. He uses the axioms required to derive the useful results he aimed for, given his approach to formalizing decision problems, and no more than that. Do you reject the plausibility of one of his axioms, or do you disagree that his results follow from his axioms?
I don’t think Peterson denies the usefulness of traditional axiomatic decision theory for checking the consistency of one’s preferences, he’s just saying that it would also be nice to have a decision theory that can tell you what you should choose given what you believe and what you value. Indeed, this is what many/most people actually do when trying to make decisions “rationally,” but this norm wasn’t justified with an axiomatic approach until Peterson (as far as I can tell).
Can you give me two examples of useful results he derives from the axioms? That’ll help me target my response. (I should note that the commentary in the grandparent is targeted at the 2004 paper in the context of the other things you’ve quoted on this page; if there’s relevant material in one of the other links I probably missed it.)
Agreed. In this comment I want to differentiate between “decision theory” and a component of it, “expected utility theory” (I didn’t differentiate between them in the grandparent). The first studies how to make decisions, and the second studies a particular mathematical technique to isolate the highest scoring of a set of alternative actions. My claim is that expected utility theory is and should be silent on the design of human-appropriate utility functions, but that decision theory should include a component focused on the design of human-appropriate utility functions. That component will be primarily researched by psychologists- what makes humans happy, what do humans want, how do we align those, what common mistakes do humans make, what intuitions do humans have and when are those useful, and so on.
Peterson’s axioms look to me like trying to shoehorn human-appropriate utility functions into expected utility theory, which doesn’t seem to augment the math of calculating expected utilities or augment the actual design of human-appropriate utility functions. As far as I can tell, that field is too young to profit from an axiomatic approach.
But I said “profit” from axioms and you said “justified” with axioms, and those are different things. It’s not clear to me that Peterson’s axioms are useful at justifying the use of expected utility theory, and my hesitance hinges on the phrase “given what you believe and what you value” from the parent. That means that’s Peterson’s decision theory takes your beliefs and values as inputs and outputs decisions- which is exactly what traditional decision theory does, and so they look the same to me (and if they’re different, I think it’s because Peterson made his worse, not better). The underlying problem as I see it is that beliefs and values are not given, they have to be extracted- and traditional decision theory underestimated the difficulty of that extraction.
(Side note: decision theory underestimating the difficulty and decision theorists underestimating the difficulty are very different things. Indeed, it’s likely that decision theorists realized the problem was very hard, and so left it to the reader so they wouldn’t have to do it!)
Then the question is how much Peterson 2004 helps its readers extract their beliefs and values. As far as I can tell, there’s very little normative or prescriptive content.
What do you mean by “the design of human-appropriate utility functions”?
Actually, let me show you a section of Peterson (2009), which is an updated and (I think) clearer presentation of his axiomatic ex ante approach. It is a bit informal, but is mercifully succinct. (The longer, formal presentation is in Peterson 2008). Here is a PDF I made of the relevant section of Peterson (2009). It’s a bit blurry, but it’s readable.
A utility function that accurately reflects the beliefs and values of the human it’s designed for. Someone looking for guidance would get assistance in discovering what their beliefs and values about the situation are, rather than just math help and a consistency check. Similarly, someone could accidentally write a utility function that drowns them in vinegar, and it would be nice if the decision-making apparatus noticed and didn’t.
That’s my interpretation of “he’s just saying that it would also be nice to have a decision theory that can tell you what you should choose given what you believe and what you value.”
This looks like it boils down to “the utility of an act is the weighted sum of the utility of its consequences.” It’s not clear to me what good formulating it like that does, and I don’t like that axiom 4 from the 2009 version looks circular. (You’re allowed to adjust the utility of different equiprobable outcomes so long as the total utility of the act is preserved. But, uh, aren’t we trying to prove that we can calculate the utility of an act with multiple possible outcome utilities, and haven’t we only assumed that it works for acts with only one possible outcome utility?)
Was Thm 4.1 an example of a useful result?
Suggestions for replacing “terminal”:
“outcome”; “ultimate” (or “ultima” if you prefer Latin); “intrinsic” (to draw a contrast with “instrumental”); “telikos” (transliteration of the Greek for final, ultimate, terminal, or last); “endpoint”; “consequence”; “effect”; “impact”; “goal.”
Do any of those sound better to you?
Edit—slight change owing to formatting.
I like “goal”, and think I like “value” even more. Value rationality?
I’m not sure about “value” in this context. The term “value” could attach to either acts or outcomes, I think. So, if the goal is to distinguish rationality that cares about acts first from rationality that cares about outcomes first, then “value” doesn’t seem to do a very good job. Does that sound right to you, or am I missing something about the distinction that you want to draw?
That’s convinced me that “goal” is clearer than “value.”