Decision Analysis Sequence

[Vaniver]

This is the introduction (conclusion) to my decision analysis sequence. It covers (much more quickly and less completely) what you would expect to see in a semester-long course on decision making. The posts are:

1. Uncertainty: the basics of treating uncertainties as probabilities and doing Bayesian math.

2. 5 Axioms of Decision Making: the five steps /​ assumptions that form the foundation of careful decision-making.

3. Compressing Reality to Math: how to take a sticky, complicated situation and condense it down to something a calculator can solve, without feeling like you’ve left something important out.

4. Measures, Risk, Death, and War: how to deal with many similar prospects (utilities), risks of death, and adversaries.

5. Value of Information: Four Examples: how to value information-gathering activity, like tests or waiting, and incorporate it into your decision-making process.

I’d like to welcome any comments about the sequence here. What parts did I do well? What parts need work? What parts would you like to see expanded (or removed)?

One of the difficulties in posting about a topic like this is that it’s foundational: basic, but important to get right. The idea of an expected utility calculation is not new (although the approach I take here may be novel for many of you) and, like I say in the VoI post, there’s often more benefit in applying the process to examples than repeatedly talking about the process. The case studies I have access to, though, are not ones I can publish online, and I don’t think I can construct an example that would work as well as a real one. Do people have problems they would like me to analyze with this framework as examples?

[fiddlemath]

Wait, really? This is only slightly incomplete from a semester-long course? This is either a strong overstatement, or seriously damning to anyone running such courses. Do they cover lots of details omitted here, or do they do lots of practice examples, or what?

Don’t get me wrong—these are really good posts! Rather, I’m kind of horrified that one could really cover a semester’s material in about two hours careful reading. If that’s the case, then college is immensely wasteful.

[Vaniver]

Rather, I’m kind of horrified that one could really cover a semester’s material in about two hours careful reading. If that’s the case, then college is immensely wasteful.

I felt the class I took was longer than it needed to be, but I also came to it with a background in economics and wargaming. I thought it was between a third and half as difficult as the average course I had taken. A lot of people haven’t thought much /​ at all about these sorts of issues, and so the class tried to be gentle to them. (It took me a few hours to read through the textbook, and I felt that contained the majority of the value the standard student would get from the class.)

Do they cover lots of details omitted here, or do they do lots of practice examples, or what?

Both, but “lots” needs to be elaborated. We spent 2-4 lectures (out of ~30) on Bayes’ Rule and heuristics and biases, which I didn’t touch on here (for obvious reasons). We spent at least one lecture on calibration, and probably another 1-2 on elicitation. (Elicitation is a big deal, and I just mentioned a few tips, but my understanding of it is not at a stage where I think I can articulate useful general advice.) We went over the mathematical aspects of utility functions and modeling in detail, whereas I mostly just mentioned names. We did several full case studies and lots of smaller examples. We did a lot of spreadsheet manipulation. We talked about presentation of results (and a few of us presented our case study reports), which is important to consultants but not as much to individuals.

Of those, the main value is probably in the practice examples.

[beoShaffer]

(It took me a few hours to read through the textbook, and I felt that contained the majority of the value the standard student would get from the class.)

What textbook did you use? Do you know how it compares to other textbooks on the subject? Also, does it include practice problem with answers? It sounds like it could be a useful recourse for those who want more detail, but can’t or won’t take a full class on it.

[Vaniver]

Unfortunately, the textbook we used was an unpublished manuscript, and didn’t have anything in the way of problems (one of the reasons I was able to finish it so quickly!). Other books I’m aware of on the subject:

Rational Decisions by Binmore is fairly close, but it focuses much more on the philosophical underpinnings of Bayesian decision theory, and is very much not a textbook (it doesn’t have problems, and mathematical sections are explicitly marked as optional).

Smart Choices by Hammond, Keeney, and Raiffa is very similar to this sequence, though they spend more time on determining preferences (which is useful for search problems, like deciding the best job to take or place to live, as they may have many different values that you need to aggregate somehow). It also doesn’t have anything in the way of problems, but has more examples.

In writing this comment, I came across Introduction to Decision Analysis by David Skinner. It appears to be textbook length, but as I haven’t read it I can’t recommend it.

Much later edit: I read and reviewed Thinking and Deciding.

2013 edit: My current favorite book on decision-making is Decisive by Heath and Heath, but it talks about habits that lead to better decisions; it is not at all a textbook. I don’t think it even mentions expected value calculation, except perhaps in passing.

[torekp]

I would like to see some exploration of decision-making frameworks built on weaker axiom sets. I mentioned this example in my reply to your second post. I don’t know enough about the subject to do the job myself.

[Vaniver]

I read that the first time around (thanks for linking it, btw) but have not given it enough thought to make a serious response. It’s on my list of things I’d like to do.