Really great discussion here, on an important and action-guiding question.
I’m confused about some of the discussion of predicting impact.
If we’re dealing with a power-law, then most of the variance in impact comes from a handful of samples. So if you’re using a metric like “contrarianness+conscientiuosness” that corresponds to an exceedingly rare trait, it might look like you’re predictions are awful, because thousands of CEOs and career executives who are succesful by common standards lack that trait. However, as long as you get Musk and a handful others right, you will have correctly predicted most of the impact, despite missing most of the succesful people. What matters is not how many data-points you get right, but which ones.
Similarly, were it the case that one or two tail-end individuals (like Warren Buffett) score within 2 standard deviations on IQ, that would make IQ a substantially worse metric for predicting who will have the most impact. I haven’t found any such individual, but I think doing so suffices to discredit some of the psychometric study conclusions as long as they didn’t include that particular individual (which they likely didn’t).
I haven’t found any such individual, but I think doing so suffices to discredit some of the psychometric study conclusions as long as they didn’t include that particular individual
Well, only if the individual would falsify the study. My claim is that the folks at the end of the power law will have these properties. I think of it as a filtering mechanism: first you filter by the first order factors, then the second order, and so on, each one doing less work than the last (for example, filtering by >2 SD IQ will cut you to <5% of the population, but once you’re just down to the best 0.01% then the third order factors will help you pick out the peak, even though those factors wouldn’t have cut down the world very much to start with).
Really great discussion here, on an important and action-guiding question.
I’m confused about some of the discussion of predicting impact.
If we’re dealing with a power-law, then most of the variance in impact comes from a handful of samples. So if you’re using a metric like “contrarianness+conscientiuosness” that corresponds to an exceedingly rare trait, it might look like you’re predictions are awful, because thousands of CEOs and career executives who are succesful by common standards lack that trait. However, as long as you get Musk and a handful others right, you will have correctly predicted most of the impact, despite missing most of the succesful people. What matters is not how many data-points you get right, but which ones.
Similarly, were it the case that one or two tail-end individuals (like Warren Buffett) score within 2 standard deviations on IQ, that would make IQ a substantially worse metric for predicting who will have the most impact. I haven’t found any such individual, but I think doing so suffices to discredit some of the psychometric study conclusions as long as they didn’t include that particular individual (which they likely didn’t).
Well, only if the individual would falsify the study. My claim is that the folks at the end of the power law will have these properties. I think of it as a filtering mechanism: first you filter by the first order factors, then the second order, and so on, each one doing less work than the last (for example, filtering by >2 SD IQ will cut you to <5% of the population, but once you’re just down to the best 0.01% then the third order factors will help you pick out the peak, even though those factors wouldn’t have cut down the world very much to start with).