If you ask, including in the Royal Society, ‘what proportion of kids with an IQ of X could master integration given a great teacher?’, you will get only blank looks and ‘I don’t think anyone has researched that’.
I think a better question is to ask: “How do you teach integration in a way that a lot of children can understand it for a reasonable time investment.
Not to mention, I strongly suspect that within the current educational system, the limiting factor is conscientiousness, not IQ. (At least, for high school and most undergraduate systems).
It’s true, few people really get integration, but I know plenty of people who are capable of memorizing integration rules sufficiently well to succeed academically—when you get down to it, integral(ax^n)dx=[a/(n+1)]*x^(n+1) shouldn’t be hard for a sufficiently conscientious person to memorize and apply. Actually understanding what’s going on and being able to derive that from scratch is just an obscure bonus accomplishment.
Why do I bring this up? Memorized integration probably isn’t worth the time investment outside of academic success, while an intuitive understanding of integration probably is worth the time investment. If you don’t design the indicators of learning carefully, you’ll end up optimizing for the wrong thing.
My anecdotal evidence runs opposite yours, and I suspect the people you are thinking of are either both smart *and conscientious, or happen to be in one of those rare academic setting which favor intelligence over conscientiousness. Or maybe your perception of peoples intelligence is influenced by how academically successful you know them to be. Who knows how perceptions of intelligence map on to IQ and GPA conscientiousness when humans use funny metrics like facial symmetry, clothing, facial expressions, eye contact and other forms of nonverbal body language?
Summarizing from my general impression of things I’ve read, IQ is generally more predictive than self-reported conscientiousness for grades …but the difference is way closer then you’d expect for a self report going up against a cognitive test.
a lot of us can attest
Hmm...so you’re drawing from the LW demographic? That’s gonna be a higher IQ group, but they’ll also have other unusual traits...I’m kind of curious now. Good candidate question for the next census...
Edit: Here’s a poll with lots of relevant data (on GPA , IQ, and procrastination of Lesswrongers). Analysis not geared towards our specific question. Raw data is available. Eyeballing it, LW GPA’s tend to be 3.3-3.9, IQ’s range 130-160. Apparently IQ correlates with absolutely none of these things...which is actually not that unexpected, since everyone is in the higher range and they probably didn’t all use the same test. But still, it doesn’t seem like LWers are skating through school...just hovering above average. And this is just the out of the people that volunteered the self-report GPA. Self reported procrastination in childhood did correlate with self reported low high school GPA,
Actually academic achievement is something IQ tests excel at predicting quite well. We also, on average, see clear differences in average intelligence between people with differences in academic achievement, if one was tempted to dismiss the greatest achievement of psychometrics out of hand based on this. This is not to say that conscientiousness isn’t another thing the educational system selects for, it is the second best predictor, but it is just that, second best. Formal education taken as a whole from primary school to grad seems to be primarily g loaded.
I dislike the ideological side effects of the current educational system, but it seems far from obvious that formal education should value raw intelligence even more over conscientiousness. Neither is something people have control over, so the question is what is the economic value of both and which kind of virtues, and by extension the people who undeserving hold them, do we wish to celebrate.
Actually academic achievement is something IQ tests excel at predicting quite well. We also, on average, see clear differences in average intelligence between people with differences in academic achievement, if one was tempted to dismiss the greatest achievement of psychometrics out of hand based on this.
I don’t deny that. I’m not one of those IQ-doesn’t-measure-anything people.
conscientiousness isn’t another thing the educational system selects for, it is the second best predictor, but it is just that, second best.
Okay, so: When you measure a correlation, you aren’t just measuring how two things are related. Construct validity plays a huge role.
If you just asked people “How smart are you” and correlated it with grades, you’d likely see a positive correlation. But if you give them Raven’s Progressive Matrices, you’d see a much stronger correlation with grades.
The correlation reflects not just the relationships between underlying phenomenon, but the degree to which you have successfully measured the underlying phenomenon. Unless you’re measuring opinions or something, self-reports suffer from all sorts of issues with validity that cognitive tests do not.
So when you compare simple self-reported conscientiousness to IQ (which is, as you said, the greatest achievement in psycho-metrics), you’re pitting a mouse against a lion.
The study I cited further down the thread, which says that willpower is more important than IQ, was able to get that result because they put a lot more effort into measuring willpower than other studies. The willpower variable was a composite of several self reports, teacher reports, parent reports, and a behavioral delay of gratification task. This composite willpower score will have more validity than any of its individual components.
(It’s the same with IQ: a composite IQ test, with verbal tests, visuospatial tests, reaction time, etc will be more g-loaded (against a separate test battery,) than any individual measurement, and will probably predict grades better too. Excuse my glossing over things—see here for an example of how CCFT’s low question diversity results in lower g)
Don’t be too quick to downplay the importance of conscientiousness—a lot of the weaker correlations can be chalked up to the difficulty of measuring the underlying thing.
If you just asked people “How smart are you” and correlated it with grades, you’d likely see a positive correlation. But if you give them Raven’s Progressive Matrices, you’d see a much stronger correlation with grades.
That’s not self-evident to me, if only because people’s self-perception of smartness is likely to be (partially) driven precisely by their grades.
I agree that its very plausible that grades affect self perception. However, I have low confidence that it would be more predictive than IQ.
Same could be said for conscientiousness, but thus far it seems like the behavioral testing and the reports from grade-naive “homeroom” teachers seem to increase the correlation rather than decrease it compared to self-report alone.
I don’t have strong opinions on the subject, but I wouldn’t necessarily expect uniform results across the IQ spectrum. It might well be that different things are more predictive at different ends of the IQ curve.
In particular with respect to IQ and conscientiousness, it seems to me that at high IQ levels the “necessary-for-A+” IQ tops out and conscientiousness starts to dominate, while it’s the reverse with low IQ—if you’re just not smart enough, conscientiousness won’t help much.
it seems to me that at high IQ levels the “necessary-for-A+” IQ tops out and conscientiousness starts to dominate, while it’s the reverse with low IQ—if you’re just not smart enough, conscientiousness won’t help much.
I low-confidence-agree with you. That seems gut-level correct, and fig 1 supports this notion but I don’t know the extent that trend should be trusted and there are other possibilities.
If I had to guess I’d rephrase it as ” both have diminishing returns, but the diminishing returns on IQ are both more dramatic than those on conscientiousness.”
actually, looking at figure 1 it doesn’t look like self discipline has diminishing returns at all, whereas it looks like IQ does. Just eyeballing, the “IQ test ceiling” for GPA is the 3rd quintile of this population’s OLSAT7 lv G scorers whereas Self Discipline doesn’t even hit a ceiling. But conclusions via eyeballing would be criminally non-rigorous :)
Not to mention, I strongly suspect that within the current educational system, the limiting factor is conscientiousness, not IQ. (At least, for high school and most undergraduate systems).
You are right, instead of focusing on teaching integration better, focusing on teaching conscientiousness might be the better goal.
Assuming that it can be taught. It’s possible that the key components that link conscientiousness to academic success are prefrontal/ventral tegmental stuff such as different reward mechanisms (motivation, willpower) and differences in executive function (attention and inhibitory control), and we don’t know to what extent teaching can modify that. Executive function, at least, seems difficult to alter via training.
Plus, we’d have to separate out the components of conscientiousness though—it’s possible that conscientiousness is not entirely positive. Conscientiousness has a rather controversial relationship with fluid intelligence and creativity, (as usual, most stereotypes carry at least some distorted truth) though it’s too soon to make definitive statements.
I do mostly agree with you—these are just rather important qualifiers.
One of the most interesting parts of Science and Sanity (Alfred Korzybski, Map is Not the Territory originator) was the analysis of mathematics and physics, and in particular “On the Semantics of the Differential Calculus”.
A better and more intuitive way to talk about Calculus, IMO.
I imagine that things like this could be experimentally measured using Khan Academy style education.
Ask teams of teachers to create different video lessons explaining X. Take a lot of students, assign them randomly to different lessons. After lessons, give them tests. Measure how good they were at tests. Choose the best lessons and reward the authors. Use those lessons for education, and once in a while announce a new competition.
Unfortunately, teaching in person cannot be replicated as well as video teaching. It is difficult to copy a teacher, so even if one teacher has a lot of success using some specific method, it does not mean others will have the same success when trying the same thing.
Ask teams of teachers to create different video lessons explaining X.
That accepts basic premises about education that I don’t think make sense. I don’t think that there any basis to believe that video explaining something are a time effective way of learning something. The same goes for straight lecturing of information.
Children don’t learn their native language because their parents explain them how it works.
On the level of making education policy the easist thing would be to simple get rid of the curriculum and let every school teach what they consider to make sense. Additionally you facillitate lots of knowledge exchange between teachers.
That would allow innovation. It might even allow for children spending more time in front of Khan Academy.
It is difficult to copy a teacher, so even if one teacher has a lot of success using some specific method, it does not mean others will have the same success when trying the same thing.
That reminds me of Aikido grandmaster Koichi Tohei who made the point that the really important skill is to teach people who teach teachers effectively.
Even if video teaching is (today?) not the best way, I think it would be nice to create a feedback loop, because feedback is what’s missing in education now. Once you have a system “here is a video, students see it, students take tests”, you can experiment with various changes and see whether those changes improved the results. The same thing could be done with books, of course. The important part is to allow the education to replicate, and measure its results. Then a way to gradual improvement (instead of random drift) is opened.
These days, education is typically done like this: random people create new educational theories mostly based on pseudoscience, teachers are taught these theories, teachers do random things in classrooms, nobody really evaluates what’s going on as long as nothing extreme happens. -- In this situation if you ask questions like “what are the different known ways to teach integrals, and how efficient is each of them”, no one really knows, because no one ever measured that in any meaningful way. The only answer you could get is the latest fashion in pseudoscience, for example “you should support multiple learning styles and, uhm, make it funny”, which, even if you’d happen to agree with it, is not specific enough to give measurable results.
It is certainly not the only way to teach, but it is a way that could be measured. We should at least try it experimentally, so see what kinds of results it could produce.
On the level of making education policy the easist thing would be to simple get rid of the curriculum and let every school teach what they consider to make sense.
And measure the outcomes, and reward those who have the best ones. The rewards can be financial, but also the prestige. (If you have a private school, and the state makes everyone known that you are the best school in the country, you are going to get a lot of money even without the state giving it to you.)
the really important skill is to teach people who teach teachers effectively.
Going more meta! But the question of measuring is still here.
And measure the outcomes, and reward those who have the best ones.
That leaves the question about which outcomes you measure. I think it’s okay to have a world with some school that run like KIPP where there a lot of measurement and others that run like Sudbury Valley with has feedback principle like internal elections and reviews how many of it’s student succeed at college.
When it comes to the specific example of teaching integration I think that will be done best via some computer tool.
I guess that 20 hours of time investment into practing a well developed Anki deck on integration would leave most students with more knowledge of integration afterwards.
I think a better question is to ask: “How do you teach integration in a way that a lot of children can understand it for a reasonable time investment.
Not to mention, I strongly suspect that within the current educational system, the limiting factor is conscientiousness, not IQ. (At least, for high school and most undergraduate systems).
It’s true, few people really get integration, but I know plenty of people who are capable of memorizing integration rules sufficiently well to succeed academically—when you get down to it, integral(ax^n)dx=[a/(n+1)]*x^(n+1) shouldn’t be hard for a sufficiently conscientious person to memorize and apply. Actually understanding what’s going on and being able to derive that from scratch is just an obscure bonus accomplishment.
Why do I bring this up? Memorized integration probably isn’t worth the time investment outside of academic success, while an intuitive understanding of integration probably is worth the time investment. If you don’t design the indicators of learning carefully, you’ll end up optimizing for the wrong thing.
Not really. I think a lot of us can attest that it is pretty easy to skate by on IQ alone, and even get decent grades while doing it.
My anecdotal evidence runs opposite yours, and I suspect the people you are thinking of are either both smart *and conscientious, or happen to be in one of those rare academic setting which favor intelligence over conscientiousness. Or maybe your perception of peoples intelligence is influenced by how academically successful you know them to be. Who knows how perceptions of intelligence map on to IQ and GPA conscientiousness when humans use funny metrics like facial symmetry, clothing, facial expressions, eye contact and other forms of nonverbal body language?
My opinion isn’t just based on anecdote, though:
Various measures of self discipline twice as predictive of GPA as IQ in adolescents—and you’ll notice the importance of IQ really drops off after the middle quintile.
Self-Reported Conscientiousness predicts college GPA better than both SAT & High School GPA
Summarizing from my general impression of things I’ve read, IQ is generally more predictive than self-reported conscientiousness for grades …but the difference is way closer then you’d expect for a self report going up against a cognitive test.
Hmm...so you’re drawing from the LW demographic? That’s gonna be a higher IQ group, but they’ll also have other unusual traits...I’m kind of curious now. Good candidate question for the next census...
Edit: Here’s a poll with lots of relevant data (on GPA , IQ, and procrastination of Lesswrongers). Analysis not geared towards our specific question. Raw data is available. Eyeballing it, LW GPA’s tend to be 3.3-3.9, IQ’s range 130-160. Apparently IQ correlates with absolutely none of these things...which is actually not that unexpected, since everyone is in the higher range and they probably didn’t all use the same test. But still, it doesn’t seem like LWers are skating through school...just hovering above average. And this is just the out of the people that volunteered the self-report GPA. Self reported procrastination in childhood did correlate with self reported low high school GPA,
http://lesswrong.com/lw/7s4/poll_results_lw_probably_doesnt_cause_akrasia/
Quite possibly true, but “a lot of LessWrongers” does not an argument about the mass of the population make.
Actually academic achievement is something IQ tests excel at predicting quite well. We also, on average, see clear differences in average intelligence between people with differences in academic achievement, if one was tempted to dismiss the greatest achievement of psychometrics out of hand based on this. This is not to say that conscientiousness isn’t another thing the educational system selects for, it is the second best predictor, but it is just that, second best. Formal education taken as a whole from primary school to grad seems to be primarily g loaded.
I dislike the ideological side effects of the current educational system, but it seems far from obvious that formal education should value raw intelligence even more over conscientiousness. Neither is something people have control over, so the question is what is the economic value of both and which kind of virtues, and by extension the people who undeserving hold them, do we wish to celebrate.
I don’t deny that. I’m not one of those IQ-doesn’t-measure-anything people.
Okay, so: When you measure a correlation, you aren’t just measuring how two things are related. Construct validity plays a huge role.
If you just asked people “How smart are you” and correlated it with grades, you’d likely see a positive correlation. But if you give them Raven’s Progressive Matrices, you’d see a much stronger correlation with grades.
The correlation reflects not just the relationships between underlying phenomenon, but the degree to which you have successfully measured the underlying phenomenon. Unless you’re measuring opinions or something, self-reports suffer from all sorts of issues with validity that cognitive tests do not.
So when you compare simple self-reported conscientiousness to IQ (which is, as you said, the greatest achievement in psycho-metrics), you’re pitting a mouse against a lion.
The study I cited further down the thread, which says that willpower is more important than IQ, was able to get that result because they put a lot more effort into measuring willpower than other studies. The willpower variable was a composite of several self reports, teacher reports, parent reports, and a behavioral delay of gratification task. This composite willpower score will have more validity than any of its individual components.
(It’s the same with IQ: a composite IQ test, with verbal tests, visuospatial tests, reaction time, etc will be more g-loaded (against a separate test battery,) than any individual measurement, and will probably predict grades better too. Excuse my glossing over things—see here for an example of how CCFT’s low question diversity results in lower g)
Don’t be too quick to downplay the importance of conscientiousness—a lot of the weaker correlations can be chalked up to the difficulty of measuring the underlying thing.
That’s not self-evident to me, if only because people’s self-perception of smartness is likely to be (partially) driven precisely by their grades.
I agree that its very plausible that grades affect self perception. However, I have low confidence that it would be more predictive than IQ.
Same could be said for conscientiousness, but thus far it seems like the behavioral testing and the reports from grade-naive “homeroom” teachers seem to increase the correlation rather than decrease it compared to self-report alone.
I don’t have strong opinions on the subject, but I wouldn’t necessarily expect uniform results across the IQ spectrum. It might well be that different things are more predictive at different ends of the IQ curve.
In particular with respect to IQ and conscientiousness, it seems to me that at high IQ levels the “necessary-for-A+” IQ tops out and conscientiousness starts to dominate, while it’s the reverse with low IQ—if you’re just not smart enough, conscientiousness won’t help much.
I low-confidence-agree with you. That seems gut-level correct, and fig 1 supports this notion but I don’t know the extent that trend should be trusted and there are other possibilities.
If I had to guess I’d rephrase it as ” both have diminishing returns, but the diminishing returns on IQ are both more dramatic than those on conscientiousness.”
actually, looking at figure 1 it doesn’t look like self discipline has diminishing returns at all, whereas it looks like IQ does. Just eyeballing, the “IQ test ceiling” for GPA is the 3rd quintile of this population’s OLSAT7 lv G scorers whereas Self Discipline doesn’t even hit a ceiling. But conclusions via eyeballing would be criminally non-rigorous :)
You are right, instead of focusing on teaching integration better, focusing on teaching conscientiousness might be the better goal.
Assuming that it can be taught. It’s possible that the key components that link conscientiousness to academic success are prefrontal/ventral tegmental stuff such as different reward mechanisms (motivation, willpower) and differences in executive function (attention and inhibitory control), and we don’t know to what extent teaching can modify that. Executive function, at least, seems difficult to alter via training.
Plus, we’d have to separate out the components of conscientiousness though—it’s possible that conscientiousness is not entirely positive. Conscientiousness has a rather controversial relationship with fluid intelligence and creativity, (as usual, most stereotypes carry at least some distorted truth) though it’s too soon to make definitive statements.
I do mostly agree with you—these are just rather important qualifiers.
When we talk about training executive function through principles such as Dual ‘n’ Back we are talking about relatively little time investment.
A child spends years in school much longer than your average psychology study runs.
One of the most interesting parts of Science and Sanity (Alfred Korzybski, Map is Not the Territory originator) was the analysis of mathematics and physics, and in particular “On the Semantics of the Differential Calculus”.
A better and more intuitive way to talk about Calculus, IMO.
I imagine that things like this could be experimentally measured using Khan Academy style education.
Ask teams of teachers to create different video lessons explaining X. Take a lot of students, assign them randomly to different lessons. After lessons, give them tests. Measure how good they were at tests. Choose the best lessons and reward the authors. Use those lessons for education, and once in a while announce a new competition.
Unfortunately, teaching in person cannot be replicated as well as video teaching. It is difficult to copy a teacher, so even if one teacher has a lot of success using some specific method, it does not mean others will have the same success when trying the same thing.
That accepts basic premises about education that I don’t think make sense. I don’t think that there any basis to believe that video explaining something are a time effective way of learning something. The same goes for straight lecturing of information.
Children don’t learn their native language because their parents explain them how it works.
On the level of making education policy the easist thing would be to simple get rid of the curriculum and let every school teach what they consider to make sense. Additionally you facillitate lots of knowledge exchange between teachers.
That would allow innovation. It might even allow for children spending more time in front of Khan Academy.
That reminds me of Aikido grandmaster Koichi Tohei who made the point that the really important skill is to teach people who teach teachers effectively.
Even if video teaching is (today?) not the best way, I think it would be nice to create a feedback loop, because feedback is what’s missing in education now. Once you have a system “here is a video, students see it, students take tests”, you can experiment with various changes and see whether those changes improved the results. The same thing could be done with books, of course. The important part is to allow the education to replicate, and measure its results. Then a way to gradual improvement (instead of random drift) is opened.
These days, education is typically done like this: random people create new educational theories mostly based on pseudoscience, teachers are taught these theories, teachers do random things in classrooms, nobody really evaluates what’s going on as long as nothing extreme happens. -- In this situation if you ask questions like “what are the different known ways to teach integrals, and how efficient is each of them”, no one really knows, because no one ever measured that in any meaningful way. The only answer you could get is the latest fashion in pseudoscience, for example “you should support multiple learning styles and, uhm, make it funny”, which, even if you’d happen to agree with it, is not specific enough to give measurable results.
It is certainly not the only way to teach, but it is a way that could be measured. We should at least try it experimentally, so see what kinds of results it could produce.
And measure the outcomes, and reward those who have the best ones. The rewards can be financial, but also the prestige. (If you have a private school, and the state makes everyone known that you are the best school in the country, you are going to get a lot of money even without the state giving it to you.)
Going more meta! But the question of measuring is still here.
That’s the huge advantage of online learning. Performance and behavioral metrics can be as detailed as you like.
That leaves the question about which outcomes you measure. I think it’s okay to have a world with some school that run like KIPP where there a lot of measurement and others that run like Sudbury Valley with has feedback principle like internal elections and reviews how many of it’s student succeed at college.
When it comes to the specific example of teaching integration I think that will be done best via some computer tool.
I guess that 20 hours of time investment into practing a well developed Anki deck on integration would leave most students with more knowledge of integration afterwards.