The study I think you are referring to was not about whether the two humps pattern existed—that was the initial empirical observation (contrasting with other subjects such as mathematics). Instead the study was focused on a test that the authors had hoped would distinguish the two groups in advance, which was later found not to work. That in itself does not mean that the two groups don’t exist.
edit: Although I agree that the fact that someone doesn’t understand Python right away doesn’t mean they won’t be able to learn, and I don’t think Eliezer meant to imply that—it’s just an easy enough test to offer the possibility of a quick win.
Performance being bimodal doesn’t mean that aptitude is bimodal, though. Different teaching styles may have different transfer functions, as it were.
Trivially, consider a teacher who only teaches at the level of the highest-aptitude students and allows no remedial or catch-up work for the lower-level ones, versus a teacher who only teaches at the level of the lowest-aptitude students and allows no enrichment for the gifted ones. We’d expect the former to produce bimodal levels of student performance given normally-distributed aptitudes (because almost everyone is left behind and performs very poorly, while the top students excel), while the latter would tend to produce performance distributions that were flatter than the aptitude distribution (because nobody is really allowed to excel).
I think they claimed that bimodal performance was typical for CS, which if true* would require CS teaching to be systematically biased in a way that doesn’t happen in other subjects in order to produce the effect, which strikes me as possible but unlikely.
* I wish they had given more statistics on this, IIRC it was a bit anecdotal.
The study I think you are referring to was not about whether the two humps pattern existed—that was the initial empirical observation (contrasting with other subjects such as mathematics). Instead the study was focused on a test that the authors had hoped would distinguish the two groups in advance, which was later found not to work. That in itself does not mean that the two groups don’t exist.
edit: Although I agree that the fact that someone doesn’t understand Python right away doesn’t mean they won’t be able to learn, and I don’t think Eliezer meant to imply that—it’s just an easy enough test to offer the possibility of a quick win.
Performance being bimodal doesn’t mean that aptitude is bimodal, though. Different teaching styles may have different transfer functions, as it were.
Trivially, consider a teacher who only teaches at the level of the highest-aptitude students and allows no remedial or catch-up work for the lower-level ones, versus a teacher who only teaches at the level of the lowest-aptitude students and allows no enrichment for the gifted ones. We’d expect the former to produce bimodal levels of student performance given normally-distributed aptitudes (because almost everyone is left behind and performs very poorly, while the top students excel), while the latter would tend to produce performance distributions that were flatter than the aptitude distribution (because nobody is really allowed to excel).
I think they claimed that bimodal performance was typical for CS, which if true* would require CS teaching to be systematically biased in a way that doesn’t happen in other subjects in order to produce the effect, which strikes me as possible but unlikely.
* I wish they had given more statistics on this, IIRC it was a bit anecdotal.