Further thoughts, after discussion with Oli Habryka, on a model of an individual’s expected future impact:
1st order factor
Past experience of building substantial and valuable products
If someone has already done lots of the thing you’re measuring, then this is the best evidence for future success at it too
2nd order factor
IQ
This is super powerful due to the positive manifold in psychometrics, where all variables of competence correlate positively.
However, my current community which is selected fairly strongly on this—all like 2 s.d’s above average, STEM students, etc, and because the tails come apart [EDIT: also known as regressional goodheart], this only captures like 25% of the variance rather than the global ~70%. So it’s not vastly more important than some of the 3rd order factors.
3rd order factors
Conscientiousness (on Big Five)
Contrarian-ness
i.e. ability to not follow local incentives toward social conformity
4th order factor
Openness (on Big Five)
The two 3rd-order factors are interesting because they seem to anti-correlate. Conscientiousness often looks like ‘do you follow orders’ and contrarian-ness… looks like the opposite. But getting both is awesome—it’s the standard Thiel-recommendation of finding someone who is great at seemingly contradictory things.
Here are the four heuristics I mentioned in the post, and which factors they measure:
Does the person have long (>1 minute) silent pauses for thinking in their conversations?
3rd order factors: Contrarian-ness, and to a lesser extent, conscientiousness
Have they exectued long-term plans not incentivised by local environment?
1st order: Past experience of building substantial and valuable products
3rd order: contrarian-ness and conscientiousness
Contrarian beliefs form simple, communicable, predictive models with a few moving parts
“Professor Stanovich and colleagues had large samples of subjects (usually several hundred) complete judgment tests like the Linda problem, as well as an I.Q. test. The major finding was that irrationality — or what Professor Stanovich called “dysrationalia” — correlates relatively weakly with I.Q.
[...]
Based on this evidence, Professor Stanovich and colleagues have introduced the concept of the rationality quotient, or R.Q. If an I.Q. test measures something like raw intellectual horsepower (abstract reasoning and verbal ability), a test of R.Q. would measure the propensity for reflective thought — stepping back from your own thinking and correcting its faulty tendencies.
There is also now evidence that rationality, unlike intelligence, can be improved through training. [...]”
The major finding was that irrationality — or what Professor Stanovich called “dysrationalia” — correlates relatively weakly with I.Q.
RQ is predicted pretty well by IQ, correlating at 0.695 (according to Stuart Richie’s book review that I read), and it seems plausible that the rest of the variance is noise. IQ correlates positively with all important factors, and often heavily (google the ‘positive manifold’ for more info), which is why I put it so high on my list.
I conjecture that the reason why Stanovich’s research isn’t very useful, is that he tried to find some factor that was as broadly applicable to the population as IQ is. However, his assumption that IQ is missing something massive was just wrong, and so he just ended up with another measure of IQ. What would’ve been more useful would’ve been to try to find some factor that predicts success after conditioning on IQ—for example, Tetlock’s work is about figuring out how the very best people think differently than everyone else, and so his work comes out with great insights about forecasting, bayesianism and model-building.
Added: I used to be a big fan of Stanovich’s work, but when I discovered that RQ correlated with IQ at 0.7… well, that’s what caused me to realise that in fact IQ is a super great predictor of important cognitive properties. And then I read the history of IQ research, which is essentially people trying to prove as hard as they can that there are important metrics of success that don’t correlate with IQ, and then failing to do so.
Hm this is an update… I’ll have to think more about it. (The “added” section actually provided most of the force (~75%) behind my update. It’s great that you provided causal reasons for your beliefs.)
Further thoughts, after discussion with Oli Habryka, on a model of an individual’s expected future impact:
1st order factor
Past experience of building substantial and valuable products
If someone has already done lots of the thing you’re measuring, then this is the best evidence for future success at it too
2nd order factor
IQ
This is super powerful due to the positive manifold in psychometrics, where all variables of competence correlate positively.
However, my current community which is selected fairly strongly on this—all like 2 s.d’s above average, STEM students, etc, and because the tails come apart [EDIT: also known as regressional goodheart], this only captures like 25% of the variance rather than the global ~70%. So it’s not vastly more important than some of the 3rd order factors.
3rd order factors
Conscientiousness (on Big Five)
Contrarian-ness
i.e. ability to not follow local incentives toward social conformity
4th order factor
Openness (on Big Five)
The two 3rd-order factors are interesting because they seem to anti-correlate. Conscientiousness often looks like ‘do you follow orders’ and contrarian-ness… looks like the opposite. But getting both is awesome—it’s the standard Thiel-recommendation of finding someone who is great at seemingly contradictory things.
Here are the four heuristics I mentioned in the post, and which factors they measure:
Does the person have long (>1 minute) silent pauses for thinking in their conversations?
3rd order factors: Contrarian-ness, and to a lesser extent, conscientiousness
Have they exectued long-term plans not incentivised by local environment?
1st order: Past experience of building substantial and valuable products
3rd order: contrarian-ness and conscientiousness
Contrarian beliefs form simple, communicable, predictive models with a few moving parts
2nd order: IQ
3rd order: contrarian-ness
Finding insights in those they disagree with
2nd order: IQ
4th order: Openness
Have you considered/do you know more about RQ?
“Professor Stanovich and colleagues had large samples of subjects (usually several hundred) complete judgment tests like the Linda problem, as well as an I.Q. test. The major finding was that irrationality — or what Professor Stanovich called “dysrationalia” — correlates relatively weakly with I.Q.
[...]
Based on this evidence, Professor Stanovich and colleagues have introduced the concept of the rationality quotient, or R.Q. If an I.Q. test measures something like raw intellectual horsepower (abstract reasoning and verbal ability), a test of R.Q. would measure the propensity for reflective thought — stepping back from your own thinking and correcting its faulty tendencies.
There is also now evidence that rationality, unlike intelligence, can be improved through training. [...]”
https://www.nytimes.com/2016/09/18/opinion/sunday/the-difference-between-rationality-and-intelligence.html
Actually, I think this claim is wrong:
RQ is predicted pretty well by IQ, correlating at 0.695 (according to Stuart Richie’s book review that I read), and it seems plausible that the rest of the variance is noise. IQ correlates positively with all important factors, and often heavily (google the ‘positive manifold’ for more info), which is why I put it so high on my list.
I conjecture that the reason why Stanovich’s research isn’t very useful, is that he tried to find some factor that was as broadly applicable to the population as IQ is. However, his assumption that IQ is missing something massive was just wrong, and so he just ended up with another measure of IQ. What would’ve been more useful would’ve been to try to find some factor that predicts success after conditioning on IQ—for example, Tetlock’s work is about figuring out how the very best people think differently than everyone else, and so his work comes out with great insights about forecasting, bayesianism and model-building.
Added: I used to be a big fan of Stanovich’s work, but when I discovered that RQ correlated with IQ at 0.7… well, that’s what caused me to realise that in fact IQ is a super great predictor of important cognitive properties. And then I read the history of IQ research, which is essentially people trying to prove as hard as they can that there are important metrics of success that don’t correlate with IQ, and then failing to do so.
Hm this is an update… I’ll have to think more about it. (The “added” section actually provided most of the force (~75%) behind my update. It’s great that you provided causal reasons for your beliefs.)
I appreciate the feedback! Very useful to know that sort of thing.