There is a problem where I say “Your hypothesis is backed by the evidence,” when your entirely verbal theory is probably amenable to many interpretations and it’s not clear how many virtue points you should get. But, I wanted to share some things from the literature that support your points about using feelings as information and avoiding miserliness.
First, there is something that’s actually just called ‘feelings-as-information theory’, and has to do with how we, surprise, use feelings as sources of information. ‘Feelings’ is meant to be a more general term than ‘emotions.’ Some examples of feelings that happen to be classified as non-emotion feelings in this model are cognitive feelings, like surprise, or ease-of-processing/fluency experiences; moods, which are longer-term than emotions and usually involve no causal attribution; and bodily sensations, like contraction of the zygomaticus major muscles. In particular, processing fluency is used intuitively and ubiquitously as a source of information, and that’s the hot topic in that small part of cognitive science right now. I have an entire book on that one feeling. I did write about this a little bit on LW, like in Availability Heuristic Considered Ambiguous, which argues that Kahneman and Tversky’s availability heuristic can be fruitfully interpreted as a statement about the use of retrieval fluency as a source of information; and Attempts to Debias Hindsight Backfire!, which is about experiments that manipulate fluency experiences to affect people’s retroactive likelihood judgments. The idea of ‘feelings as information’ looks central to the Art.
There is also a small literature on hypothesis generation. See the section ‘Hypothesis Generation and Hypothesis Evaluation’ of this paper for a good review of everything we know about hypothesis generation. Hardly inspiring, I know. The evidence indicates that humans generate relatively few hypotheses, or we may also write, humans have impoverished hypothesis sets. Also in this paper, I saw studies that compare hypothesis generation between individuals and groups of various sizes. You’re right that groups typically generate more hypotheses than individuals. They also tried comparing ‘natural’ and ‘synthetic’ groups, natural groups are what you think; the hypothesis sets of synthetic groups are formed from the union of many individual, non-group hypothesis sets. It turns out that synthetic groups do a little better. Social interaction somehow reduces the number of alternatives that a group considers relative to what the sum of their considerations would be if they were not a group.
Also, about your planning fallacy primer, I think the memory bias account has a lot more going for it than a random individual might infer from the brevity of its discussion.
I am assuming the first point is about this post and the second two are about the planning primer?
The feelings-as-information literature is new to me, and most of what I wrote here is from conversations w/ folks at CFAR. (Who, by the way, would probably be interested in seeing those links as well.)
I’ll freely admit that the decision making part in groups was the weakest part of my planning primer. I’m not very sure on the data, so your additional info on improved group hypothesis generation is pretty cool.
There are definitely several papers on memory bias affecting decisions, although I’m unsure if we’re talking about the same thing here. What I want to say is something like “improperly recalling how long things took in the past is a problem that can bias predictions we make” and this phenomena has been studied several times.
But there is also a separate thing where “in observed studies of people planning, very few of them seem to even use their memories, in the sense of recalling past information, to create a reference class and use it to help them with their estimates for their plans”, which might also be what you’re referring to.
I am assuming the first point is about this post and the second two are about the planning primer?
The first two are about this article and the third is about the planning fallacy primer. I mentioned hypothesis generation because you talked about ‘pair debugging’ and asking people to state the obvious solutions to a problem as ways to increase the number of hypotheses that are generated, and it pattern matched to what I’d read about hypothesis generation.
There are definitely several papers on memory bias affecting decisions, although I’m unsure if we’re talking about the same thing here. What I want to say is something like “improperly recalling how long things took in the past is a problem that can bias predictions we make” and this phenomena has been studied several times.
I’m definitely talking about this as opposed to the other thing. MINERVA-DM is a good example of this class of hypothesis in the realm of likelihood judgment. Hilbert (2012) is an information-theoretic approach to memory bias in likelihood judgment.
I’m just saying that it looks like there’s a lot of fruit to be picked in memory theory and not many people are talking about it.
Okay, gotcha. Thanks for the clarification on the points.
I admit I don’t quite understand what MINERVA-DM is...I glanced at the paper briefly and it appears to be a...theoretical framework for making decisions which is shown to exhibit similar biases to human thought? (With cells and rows and ones?)
I’m definitely not strong in this domain; any chance you could summarize?
I admit I don’t quite understand what MINERVA-DM is...I glanced at the paper briefly and it appears to be a...theoretical framework for making decisions which is shown to exhibit similar biases to human thought? (With cells and rows and ones?)
I can’t describe it too much better than that. The framework is meant to be descriptive as opposed to normative.
A complete description of MINERVA-DM would involve some simple math, but I can try to describe it in words. The rows of numbers you saw are vectors. We take a vector that represents an observation, called a probe, along with all vectors in episodic memory, which are called traces, and by evaluating the similarity of the probe to each trace and averaging these similarities, we obtain a number that represents a global familiarity signal. By assuming that people use this familiarity signal as the basis of their likelihood judgments, we can simulate some of the results found in the field of likelihood judgment.
I suspect that with a bit of work, one could even use MINERVA-DM to simulate retrospective and prospective judgments of task duration, and thus, planning fallacy.
There is a problem where I say “Your hypothesis is backed by the evidence,” when your entirely verbal theory is probably amenable to many interpretations and it’s not clear how many virtue points you should get. But, I wanted to share some things from the literature that support your points about using feelings as information and avoiding miserliness.
First, there is something that’s actually just called ‘feelings-as-information theory’, and has to do with how we, surprise, use feelings as sources of information. ‘Feelings’ is meant to be a more general term than ‘emotions.’ Some examples of feelings that happen to be classified as non-emotion feelings in this model are cognitive feelings, like surprise, or ease-of-processing/fluency experiences; moods, which are longer-term than emotions and usually involve no causal attribution; and bodily sensations, like contraction of the zygomaticus major muscles. In particular, processing fluency is used intuitively and ubiquitously as a source of information, and that’s the hot topic in that small part of cognitive science right now. I have an entire book on that one feeling. I did write about this a little bit on LW, like in Availability Heuristic Considered Ambiguous, which argues that Kahneman and Tversky’s availability heuristic can be fruitfully interpreted as a statement about the use of retrieval fluency as a source of information; and Attempts to Debias Hindsight Backfire!, which is about experiments that manipulate fluency experiences to affect people’s retroactive likelihood judgments. The idea of ‘feelings as information’ looks central to the Art.
There is also a small literature on hypothesis generation. See the section ‘Hypothesis Generation and Hypothesis Evaluation’ of this paper for a good review of everything we know about hypothesis generation. Hardly inspiring, I know. The evidence indicates that humans generate relatively few hypotheses, or we may also write, humans have impoverished hypothesis sets. Also in this paper, I saw studies that compare hypothesis generation between individuals and groups of various sizes. You’re right that groups typically generate more hypotheses than individuals. They also tried comparing ‘natural’ and ‘synthetic’ groups, natural groups are what you think; the hypothesis sets of synthetic groups are formed from the union of many individual, non-group hypothesis sets. It turns out that synthetic groups do a little better. Social interaction somehow reduces the number of alternatives that a group considers relative to what the sum of their considerations would be if they were not a group.
Also, about your planning fallacy primer, I think the memory bias account has a lot more going for it than a random individual might infer from the brevity of its discussion.
Hey Gram,
Thanks for the additional information!
I am assuming the first point is about this post and the second two are about the planning primer?
The feelings-as-information literature is new to me, and most of what I wrote here is from conversations w/ folks at CFAR. (Who, by the way, would probably be interested in seeing those links as well.)
I’ll freely admit that the decision making part in groups was the weakest part of my planning primer. I’m not very sure on the data, so your additional info on improved group hypothesis generation is pretty cool.
There are definitely several papers on memory bias affecting decisions, although I’m unsure if we’re talking about the same thing here. What I want to say is something like “improperly recalling how long things took in the past is a problem that can bias predictions we make” and this phenomena has been studied several times.
But there is also a separate thing where “in observed studies of people planning, very few of them seem to even use their memories, in the sense of recalling past information, to create a reference class and use it to help them with their estimates for their plans”, which might also be what you’re referring to.
The first two are about this article and the third is about the planning fallacy primer. I mentioned hypothesis generation because you talked about ‘pair debugging’ and asking people to state the obvious solutions to a problem as ways to increase the number of hypotheses that are generated, and it pattern matched to what I’d read about hypothesis generation.
I’m definitely talking about this as opposed to the other thing. MINERVA-DM is a good example of this class of hypothesis in the realm of likelihood judgment. Hilbert (2012) is an information-theoretic approach to memory bias in likelihood judgment.
I’m just saying that it looks like there’s a lot of fruit to be picked in memory theory and not many people are talking about it.
Okay, gotcha. Thanks for the clarification on the points.
I admit I don’t quite understand what MINERVA-DM is...I glanced at the paper briefly and it appears to be a...theoretical framework for making decisions which is shown to exhibit similar biases to human thought? (With cells and rows and ones?)
I’m definitely not strong in this domain; any chance you could summarize?
I can’t describe it too much better than that. The framework is meant to be descriptive as opposed to normative.
A complete description of MINERVA-DM would involve some simple math, but I can try to describe it in words. The rows of numbers you saw are vectors. We take a vector that represents an observation, called a probe, along with all vectors in episodic memory, which are called traces, and by evaluating the similarity of the probe to each trace and averaging these similarities, we obtain a number that represents a global familiarity signal. By assuming that people use this familiarity signal as the basis of their likelihood judgments, we can simulate some of the results found in the field of likelihood judgment.
I suspect that with a bit of work, one could even use MINERVA-DM to simulate retrospective and prospective judgments of task duration, and thus, planning fallacy.
Huh, okay, cool. Thanks for the additional info!