Decision Analysis Sequence
This is the introduction (conclusion) to my decision analysis sequence. It covers (much more quickly and less completely) what you would expect to see in a semester-long course on decision making. The posts are:
Uncertainty: the basics of treating uncertainties as probabilities and doing Bayesian math.
5 Axioms of Decision Making: the five steps / assumptions that form the foundation of careful decision-making.
Compressing Reality to Math: how to take a sticky, complicated situation and condense it down to something a calculator can solve, without feeling like you’ve left something important out.
Measures, Risk, Death, and War: how to deal with many similar prospects (utilities), risks of death, and adversaries.
Value of Information: Four Examples: how to value information-gathering activity, like tests or waiting, and incorporate it into your decision-making process.
I’d like to welcome any comments about the sequence here. What parts did I do well? What parts need work? What parts would you like to see expanded (or removed)?
One of the difficulties in posting about a topic like this is that it’s foundational: basic, but important to get right. The idea of an expected utility calculation is not new (although the approach I take here may be novel for many of you) and, like I say in the VoI post, there’s often more benefit in applying the process to examples than repeatedly talking about the process. The case studies I have access to, though, are not ones I can publish online, and I don’t think I can construct an example that would work as well as a real one. Do people have problems they would like me to analyze with this framework as examples?
- Value of Information: Four Examples by Nov 22, 2011, 11:02 PM; 137 points) (
- 5 Axioms of Decision Making by Dec 1, 2011, 10:22 PM; 50 points) (
- Compressing Reality to Math by Dec 15, 2011, 12:07 AM; 34 points) (
- Uncertainty by Nov 29, 2011, 11:12 PM; 27 points) (
- Jun 11, 2012, 7:07 PM; 25 points) 's comment on Intellectual insularity and productivity by (
- Value-Focused Thinking: a chapter-by-chapter summary by Jan 23, 2013, 2:39 AM; 22 points) (
- Measures, Risk, Death, and War by Dec 20, 2011, 11:37 PM; 17 points) (
- Apr 11, 2013, 1:50 AM; 17 points) 's comment on Post Request Thread by (
- Aug 31, 2012, 4:55 PM; 14 points) 's comment on Dealing with trolling and the signal to noise ratio by (
- Feb 18, 2012, 2:31 PM; 6 points) 's comment on Mini-review: ‘Judgment and Decision Making as a Skill’ by (
- LW4EA: Value of Information: Four Examples by Jun 14, 2022, 2:26 AM; 5 points) (EA Forum;
- Nov 15, 2014, 12:31 AM; 5 points) 's comment on The “best” mathematically-informed topics? by (
- Dec 23, 2011, 5:20 PM; 4 points) 's comment on Applied Rationality Practice by (
- Jun 29, 2015, 4:45 PM; 4 points) 's comment on Two-boxing, smoking and chewing gum in Medical Newcomb problems by (
- Sep 15, 2013, 1:21 PM; 3 points) 's comment on College courses versus LessWrong by (
- May 12, 2012, 12:08 AM; 2 points) 's comment on Most transferable skills? by (
- Oct 15, 2013, 9:23 PM; 1 point) 's comment on How habits work and how you may control them by (
- Meetup : West LA Meetup—Value of Information by May 22, 2012, 4:50 AM; 1 point) (
Wait, really? This is only slightly incomplete from a semester-long course? This is either a strong overstatement, or seriously damning to anyone running such courses. Do they cover lots of details omitted here, or do they do lots of practice examples, or what?
Don’t get me wrong—these are really good posts! Rather, I’m kind of horrified that one could really cover a semester’s material in about two hours careful reading. If that’s the case, then college is immensely wasteful.
I felt the class I took was longer than it needed to be, but I also came to it with a background in economics and wargaming. I thought it was between a third and half as difficult as the average course I had taken. A lot of people haven’t thought much / at all about these sorts of issues, and so the class tried to be gentle to them. (It took me a few hours to read through the textbook, and I felt that contained the majority of the value the standard student would get from the class.)
Both, but “lots” needs to be elaborated. We spent 2-4 lectures (out of ~30) on Bayes’ Rule and heuristics and biases, which I didn’t touch on here (for obvious reasons). We spent at least one lecture on calibration, and probably another 1-2 on elicitation. (Elicitation is a big deal, and I just mentioned a few tips, but my understanding of it is not at a stage where I think I can articulate useful general advice.) We went over the mathematical aspects of utility functions and modeling in detail, whereas I mostly just mentioned names. We did several full case studies and lots of smaller examples. We did a lot of spreadsheet manipulation. We talked about presentation of results (and a few of us presented our case study reports), which is important to consultants but not as much to individuals.
Of those, the main value is probably in the practice examples.
What textbook did you use? Do you know how it compares to other textbooks on the subject? Also, does it include practice problem with answers? It sounds like it could be a useful recourse for those who want more detail, but can’t or won’t take a full class on it.
Unfortunately, the textbook we used was an unpublished manuscript, and didn’t have anything in the way of problems (one of the reasons I was able to finish it so quickly!). Other books I’m aware of on the subject:
Rational Decisions by Binmore is fairly close, but it focuses much more on the philosophical underpinnings of Bayesian decision theory, and is very much not a textbook (it doesn’t have problems, and mathematical sections are explicitly marked as optional).
Smart Choices by Hammond, Keeney, and Raiffa is very similar to this sequence, though they spend more time on determining preferences (which is useful for search problems, like deciding the best job to take or place to live, as they may have many different values that you need to aggregate somehow). It also doesn’t have anything in the way of problems, but has more examples.
In writing this comment, I came across Introduction to Decision Analysis by David Skinner. It appears to be textbook length, but as I haven’t read it I can’t recommend it.
Much later edit: I read and reviewed Thinking and Deciding.
2013 edit: My current favorite book on decision-making is Decisive by Heath and Heath, but it talks about habits that lead to better decisions; it is not at all a textbook. I don’t think it even mentions expected value calculation, except perhaps in passing.
I would like to see some exploration of decision-making frameworks built on weaker axiom sets. I mentioned this example in my reply to your second post. I don’t know enough about the subject to do the job myself.
I read that the first time around (thanks for linking it, btw) but have not given it enough thought to make a serious response. It’s on my list of things I’d like to do.