I find this super interesting, but as always I worry about selection effects.
There are many famous, successful and influential people in history. My question would be what % of those people had tutoring, cognitive apprenticeships etc...
This post chooses a number of famous people. Presumably the selection process goes something like this:
look at a list of famous people
look which ones have something written about their education
writes about those one
The problem is that those with unusual educations are more likely to have written about them. What if there are many famous/successful people who mostly had normal education
Your question seems like it should be the other way around. We don’t really care about P(tutoring | success) directly when choosing actions, but we do care about P(success | tutoring) versus P(success | not tutoring).
Unless the relevant base rates P(tutoring) and P(success) are known, P(tutoring | success) by itself tells us little.
So what I have done is altogether to rough to answer this question. But from my sample (which is basically me writing down about 30 names I can think of as exceptional and then looking at their bio), tutoring seems to have played an important part for at least 70 percent. By which I mean, they got at least an hour a day of formal tutoring from someone skilled at it. I think that is more than average.
Tutoring is not as universal as just having really smart people around to talk to, though. That is nearly universal in my sample, and is surely less common among unsuccessful people.
I don’t find your methodology for deciding when tutoring has played an important part persuasive.
In fact, even if we could show that P(success | tutoring) > P(success | not tutoring), that again by itself would tell us little because it would only be correlational evidence. Judging whether tutoring played an important part in the success of these people needs to be done using a more rigorous causal analysis, which means controlling for obvious confounders such as family wealth and genetic endowment in some form even if the study has to be observational in nature. This is impossible to do simply by reading Wikipedia articles about people who have been successful.
Tutoring is not as universal as just having really smart people around to talk to, though. That is nearly universal in my sample, and is surely less common among unsuccessful people.
Again, that being less common among unsuccessful people doesn’t tell us anything of value, because
it’s only correlational evidence, not causal; and
it’s only directional evidence and doesn’t give us much information about the magnitude of the effect.
The interesting question here is about the effect size—on priors I think it’s easy to agree that having smart people to talk with during childhood would have a positive impact on your future success as an adult. However, is this a d = 0.01 effect, a d = 0.1 effect or a d = 1 effect? What’s the order of magnitude?
I would expect the effect size of childhood tutoring to be small to moderate if we could actually run this experiment or at least get good enough observational data to control for the obvious confounders, and I don’t think this position is really contradicted by the information presented in your post. As a consequence, I remain unconvinced by your central thesis.
I like your rigor—I feel too time-contained to be this systematic when I think about how to raise my kids. I would love to know how you would approach that decision—what data you would look at. And if you have kids, or know how you would raise them, I would love to know how you approach it, too. Especially the parts that contradict the patterns I noted in the sample in my essay.
There are selection effects, for sure. The process wasn’t as bad as you describe, but it was pretty bad as I describe in the post. I made the list of names (before looking up what they had written etc). I also actively looked for counterexamples to add to the list later. So the number 2⁄3′s homeschooled for example is just the number I got going through everyone. About a third did go to schools, Jesuit schools being most common—for my sample. The post itself uses a lot of colorful examples, because, that’s pretty much what I’m doing. Getting an impression.
Hmmmm.
I find this super interesting, but as always I worry about selection effects.
There are many famous, successful and influential people in history. My question would be what % of those people had tutoring, cognitive apprenticeships etc...
This post chooses a number of famous people. Presumably the selection process goes something like this:
look at a list of famous people
look which ones have something written about their education
writes about those one
The problem is that those with unusual educations are more likely to have written about them. What if there are many famous/successful people who mostly had normal education
Your question seems like it should be the other way around. We don’t really care about P(tutoring | success) directly when choosing actions, but we do care about P(success | tutoring) versus P(success | not tutoring).
Unless the relevant base rates P(tutoring) and P(success) are known, P(tutoring | success) by itself tells us little.
So what I have done is altogether to rough to answer this question. But from my sample (which is basically me writing down about 30 names I can think of as exceptional and then looking at their bio), tutoring seems to have played an important part for at least 70 percent. By which I mean, they got at least an hour a day of formal tutoring from someone skilled at it. I think that is more than average.
Tutoring is not as universal as just having really smart people around to talk to, though. That is nearly universal in my sample, and is surely less common among unsuccessful people.
I don’t find your methodology for deciding when tutoring has played an important part persuasive.
In fact, even if we could show that P(success | tutoring) > P(success | not tutoring), that again by itself would tell us little because it would only be correlational evidence. Judging whether tutoring played an important part in the success of these people needs to be done using a more rigorous causal analysis, which means controlling for obvious confounders such as family wealth and genetic endowment in some form even if the study has to be observational in nature. This is impossible to do simply by reading Wikipedia articles about people who have been successful.
Again, that being less common among unsuccessful people doesn’t tell us anything of value, because
it’s only correlational evidence, not causal; and
it’s only directional evidence and doesn’t give us much information about the magnitude of the effect.
The interesting question here is about the effect size—on priors I think it’s easy to agree that having smart people to talk with during childhood would have a positive impact on your future success as an adult. However, is this a d = 0.01 effect, a d = 0.1 effect or a d = 1 effect? What’s the order of magnitude?
I would expect the effect size of childhood tutoring to be small to moderate if we could actually run this experiment or at least get good enough observational data to control for the obvious confounders, and I don’t think this position is really contradicted by the information presented in your post. As a consequence, I remain unconvinced by your central thesis.
I like your rigor—I feel too time-contained to be this systematic when I think about how to raise my kids. I would love to know how you would approach that decision—what data you would look at. And if you have kids, or know how you would raise them, I would love to know how you approach it, too. Especially the parts that contradict the patterns I noted in the sample in my essay.
There are selection effects, for sure. The process wasn’t as bad as you describe, but it was pretty bad as I describe in the post. I made the list of names (before looking up what they had written etc). I also actively looked for counterexamples to add to the list later. So the number 2⁄3′s homeschooled for example is just the number I got going through everyone. About a third did go to schools, Jesuit schools being most common—for my sample. The post itself uses a lot of colorful examples, because, that’s pretty much what I’m doing. Getting an impression.