This is a messy subject, and one that’s difficult write about, and I appreciate you tackling the topic. I think there are some important qualifications to make about this post, as others havenoted. But I know that when writing about messy subjects, it’s hard to avoid “death by a thousand qualifications.” Lately, I’ve been trying to solve the problem by putting most qualifications in footnotes, e.g. here. You might want to try that, as it mitigates criticisms of the “but you didn’t make qualifications X and Y!” form while still leaving the body text in a readable condition.
Below, I’ll refer to MWA (“many weak arguments”) and ORSA (“one relatively strong argument”), for convenience.
Here’s my guess at what’s going on:
Probability theory doesn’t “intrinsically favor” MWA over ORSA. Both have their uses, their limits, and their “gotchas” when applied to bounded rationality. If MWA is in some important sense “reliably better” than ORSA, I’d need stronger evidence/argument than is provided in this post. (That’s not necessarily a criticism of the post; putting together strong evidence+argument about messy subjects is difficult and time-consuming.)
For historical reasons, Less Wrong tended to attract people accustomed to ORSA thinking — which is common in e.g. mathematics and philosophy. Hence, LWers tend to make “too much reliance on ORSA”-type mistakes more often than they make the “too much reliance on MWA”-type mistakes.
Givewell tends to emphasize the MWA approach, and has been remarkably successful at figuring out the parts of the world they’re trying to understand. This impressed you, and helped you to realize that, like many mathematicians, you had been placing too much emphasis on ORSA thinking.
I think there are some important qualifications to make about this post, as others have noted.
My hunch is that most significant problem with the MWA approach is the assumption of (weak) independence, in the sense that in practice, when sophisticated use of MWA fails, it’s usually because the weak lines of evidence are all being driven by the same selection effect. A hypothetical example that jumps to mind is:
A VC is evaluating a startup. He or she reasons
The sector is growing
My colleagues think that the sector is good to invest in
On an object level, their plan looks good
The people are impressive
and the situation is
Re: #1 — The reason that the sector is growing is because there’s a bubble
Re: #2 — The reason that the VC’s colleagues think that the sector is good to invest in is because, like the VC, they don’t recognize that there’s a bubble.
Re: #3 — The VC’s views on the object level merit of the project are colored by the memes that have been spreading around that are causing the bubble
Re: #4 — The reason that impressive people are going into the sector is because there’s a bubble, so everyone’s going into the sector – the people’s impressiveness isn’t manifesting itself in their choosing a good focus.
I don’t know whether this situation occurs in practice, but it seems very possible.
Givewell tends to emphasize the MWA approach, and has been remarkably successful at figuring out the parts of the world they’re trying to understand.
GiveWell is an interesting case, insofar as it’s done more ORSA work than I’ve seen in most contexts. The page on long lasting insecticide treated nets provides examples. Part of why I’m favoring MWA is because GiveWell has done both and of the two, leans toward MWA.
This is a great example. It’s often very hard to tell whether MWA are independent or not. They could all derive from the same factors. Or they could all be made up by the same type of motivated reasoning.
I think that’s the judgment of being a good “Fox” ala Tetlock’s Hedgehog vs the Fox.
Which situation? The VC startup thing? The OSRA style is to scrutinize arguments carefully to see if they break down. If you can recognize that the arguments all break down in the same way, then you can conclude that the arguments are dependent and that even collectively, they don’t constitute much evidence.
But that still doesn’t tell you whether to invest in the startup. If an ORSA-er is just paralyzed by indecision here and decides to leave VC and go into theoretical math or whatever, he or she is not really winning.
Unrelatedly, a fun example of MWA triumphing over ORSA could be geologists vs. physicists on the age of the Earth.
But that still doesn’t tell you whether to invest in the startup. If an ORSA-er is just paralyzed by indecision here and decides to leave VC and go into theoretical math or whatever, he or she is not really winning.
I would guess that ORSA doesn’t suffice to be a successful VC. The claim is that it could help, in conjunction with MWAs.
If you scrutinize the weak arguments and find that the break down in different ways, then that suggests that the arguments are independent, and that you should invest in the start-up. If you find that they break down in the same way, then that suggests that you shouldn’t invest in the start-up.
For historical reasons, Less Wrong tended to attract people accustomed to ORSA thinking — which is common in e.g. mathematics and philosophy. Hence, LWers tend to make “too much reliance on ORSA”-type mistakes more often than they make the “too much reliance on MWA”-type mistakes.
Could you (or someone else) possibly give some examples of this? This seems like it’s probably true but I’m having trouble thinking of concrete examples. I want to know the nature of the bias I should be compensating for.
Probability theory doesn’t “intrinsically favor” MWA over ORSA. Both have their uses, their limits, and their “gotchas” when applied to bounded rationality. If MWA is in some important sense “reliably better” than ORSA, I’d need stronger evidence/argument than is provided in this post.
It’s true that both MWA and ORSA break down badly in certain, non overlapping, contexts.
I currently believe that MWA generally produces better predictive models about the human world than ORSA does. The context of the human world is a special one, and I would expect ORSA to systematically be better in some natural (not necessarily mathematical) contexts.
I believe that ORSA does outperform MWA in certain situations involving the human world (c.f. the remarks in my other comment about bubbles).
This is a messy subject, and one that’s difficult write about, and I appreciate you tackling the topic. I think there are some important qualifications to make about this post, as others have noted. But I know that when writing about messy subjects, it’s hard to avoid “death by a thousand qualifications.” Lately, I’ve been trying to solve the problem by putting most qualifications in footnotes, e.g. here. You might want to try that, as it mitigates criticisms of the “but you didn’t make qualifications X and Y!” form while still leaving the body text in a readable condition.
Below, I’ll refer to MWA (“many weak arguments”) and ORSA (“one relatively strong argument”), for convenience.
Here’s my guess at what’s going on:
Probability theory doesn’t “intrinsically favor” MWA over ORSA. Both have their uses, their limits, and their “gotchas” when applied to bounded rationality. If MWA is in some important sense “reliably better” than ORSA, I’d need stronger evidence/argument than is provided in this post. (That’s not necessarily a criticism of the post; putting together strong evidence+argument about messy subjects is difficult and time-consuming.)
For historical reasons, Less Wrong tended to attract people accustomed to ORSA thinking — which is common in e.g. mathematics and philosophy. Hence, LWers tend to make “too much reliance on ORSA”-type mistakes more often than they make the “too much reliance on MWA”-type mistakes.
Givewell tends to emphasize the MWA approach, and has been remarkably successful at figuring out the parts of the world they’re trying to understand. This impressed you, and helped you to realize that, like many mathematicians, you had been placing too much emphasis on ORSA thinking.
Thanks for the feedback.
My hunch is that most significant problem with the MWA approach is the assumption of (weak) independence, in the sense that in practice, when sophisticated use of MWA fails, it’s usually because the weak lines of evidence are all being driven by the same selection effect. A hypothetical example that jumps to mind is:
A VC is evaluating a startup. He or she reasons
The sector is growing
My colleagues think that the sector is good to invest in
On an object level, their plan looks good
The people are impressive
and the situation is
Re: #1 — The reason that the sector is growing is because there’s a bubble
Re: #2 — The reason that the VC’s colleagues think that the sector is good to invest in is because, like the VC, they don’t recognize that there’s a bubble.
Re: #3 — The VC’s views on the object level merit of the project are colored by the memes that have been spreading around that are causing the bubble
Re: #4 — The reason that impressive people are going into the sector is because there’s a bubble, so everyone’s going into the sector – the people’s impressiveness isn’t manifesting itself in their choosing a good focus.
I don’t know whether this situation occurs in practice, but it seems very possible.
GiveWell is an interesting case, insofar as it’s done more ORSA work than I’ve seen in most contexts. The page on long lasting insecticide treated nets provides examples. Part of why I’m favoring MWA is because GiveWell has done both and of the two, leans toward MWA.
This is a great example. It’s often very hard to tell whether MWA are independent or not. They could all derive from the same factors. Or they could all be made up by the same type of motivated reasoning.
I think that’s the judgment of being a good “Fox” ala Tetlock’s Hedgehog vs the Fox.
Do you know ORSA that gets you out of this situation?
Which situation? The VC startup thing? The OSRA style is to scrutinize arguments carefully to see if they break down. If you can recognize that the arguments all break down in the same way, then you can conclude that the arguments are dependent and that even collectively, they don’t constitute much evidence.
But that still doesn’t tell you whether to invest in the startup. If an ORSA-er is just paralyzed by indecision here and decides to leave VC and go into theoretical math or whatever, he or she is not really winning.
Unrelatedly, a fun example of MWA triumphing over ORSA could be geologists vs. physicists on the age of the Earth.
I would guess that ORSA doesn’t suffice to be a successful VC. The claim is that it could help, in conjunction with MWAs.
If you scrutinize the weak arguments and find that the break down in different ways, then that suggests that the arguments are independent, and that you should invest in the start-up. If you find that they break down in the same way, then that suggests that you shouldn’t invest in the start-up.
I’d definitely be interested to learn (in more detail, with more examples) why this is. They may very well have good reasons for it.
Could you (or someone else) possibly give some examples of this? This seems like it’s probably true but I’m having trouble thinking of concrete examples. I want to know the nature of the bias I should be compensating for.
It’s true that both MWA and ORSA break down badly in certain, non overlapping, contexts.
I currently believe that MWA generally produces better predictive models about the human world than ORSA does. The context of the human world is a special one, and I would expect ORSA to systematically be better in some natural (not necessarily mathematical) contexts.
I believe that ORSA does outperform MWA in certain situations involving the human world (c.f. the remarks in my other comment about bubbles).