“But this example also helps show the limits of VoI: VoI is best suited to situations where you’ve done the background research and are now considering further experiments.”
Do you mean this in the sense that there is usually some low-hanging fruit (e.g. the background research itself) where the VOI is obviously so high that you there’s no need to calculate it—you obviously should obtain the information?
I think Douglas Hubbard, author of How to Measure Anything, makes a good case for making VOI calculations the default in important decisions. When he acts as a consultant, he first spends a couple of days training the decision makers in calibrating their probability assessments, and then they do a VOI calculation for all the important unknowns, based on those subjective probabilities. It’s often precisely those questions for which they can’t get much from background research and haven’t even considered measuring—because they don’t know how to—that have the highest VOI.
Maybe these cases are atypical, as they are biased towards difficult decisions that warrant hiring a consultant. But difficult decisions are the raison d’etre for the field of decision analysis.
Hubbard talks about measurement inversion. “In a business case, the economic value of measuring a variable is usually inversely proportional to how much measurement attention it usually gets.” This thread contains discussion about possible reasons for it. The easiness/familiarity aspect that you imply is probably one of them. Others include declining marginal value of information on certain subject and the destabilizing effect new measurements might have for an organization.
It’s easy to imagine that measurement inversion would also apply to common measurements on personal life.
When he acts as a consultant, he first spends a couple of days training the decision makers in calibrating their probability assessments, and then they do a VOI calculation for all the important unknowns, based on those subjective probabilities.
As I’m not Vaniver I can’t say for sure, but the way I read it doing the background research refers to the step were you get the subjective probabilities and identify the important unknowns.
“But this example also helps show the limits of VoI: VoI is best suited to situations where you’ve done the background research and are now considering further experiments.”
Do you mean this in the sense that there is usually some low-hanging fruit (e.g. the background research itself) where the VOI is obviously so high that you there’s no need to calculate it—you obviously should obtain the information?
I think Douglas Hubbard, author of How to Measure Anything, makes a good case for making VOI calculations the default in important decisions. When he acts as a consultant, he first spends a couple of days training the decision makers in calibrating their probability assessments, and then they do a VOI calculation for all the important unknowns, based on those subjective probabilities. It’s often precisely those questions for which they can’t get much from background research and haven’t even considered measuring—because they don’t know how to—that have the highest VOI.
Maybe these cases are atypical, as they are biased towards difficult decisions that warrant hiring a consultant. But difficult decisions are the raison d’etre for the field of decision analysis.
Hubbard talks about measurement inversion. “In a business case, the economic value of measuring a variable is usually inversely proportional to how much measurement attention it usually gets.” This thread contains discussion about possible reasons for it. The easiness/familiarity aspect that you imply is probably one of them. Others include declining marginal value of information on certain subject and the destabilizing effect new measurements might have for an organization.
It’s easy to imagine that measurement inversion would also apply to common measurements on personal life.
As I’m not Vaniver I can’t say for sure, but the way I read it doing the background research refers to the step were you get the subjective probabilities and identify the important unknowns.