In all countries, schools are sampled, attempting to control for some variables. Around 5000 students, 160 schools in the US. Note that schools may/may not choose to participate, and the same with students. PISA has minimum standards for acceptance as sampled selection, in an attempt to avoid the obvious bias that countries would have an interest to produce in their samples.
Given the optional sampling, and obvious incentives, I’m skeptical that this is particularly accurate, and an apples to apples comparison.
Also, homeschoolers, one of the strongest demographics for ethnic and intact family reasons, are likely missed by this sample, skewing results in the US down. They appear to be about 4% of the total population, disproportionately white and in two parent households.
Note that schools may/may not choose to participate, and the same with students.
They control for schools opting not to participate; see the section under substitute schools.
It’s standard research ethics that minors (and their guardians) be given the option to refuse to participate in a study.
Also, homeschoolers, one of the strongest demographics for ethnic and intact family reasons, are likely missed by this sample, skewing results in the US down. They appear to be about 4% of the total population, disproportionately white and in two parent households.
Now I’m confused. The original inference Sailer drew from PISA was that American students outperform their racial group in other countries. You’re claiming the study will be biased against white Americans. If anything you should be annoyed at Sailer for trying to support his racial performance narrative using a study that didn’t really focus on it.
Regarding homeschooling in particular, it’d be nearly impossible to develop an international study on the same scale of PISA (which you already want to reject as too small) merely because homeschooling isn’t prevalent in most of the PISA-participating countries.
They control for schools opting not to participate; see the section under substitute schools.
I’m aware that they tried to control, but the bounds are large and open to exploitation by the countries who so choose to do so.
And the homeschool issue should strengthen his conclusion. I was just noting a factor he hadn’t controlled for.
Both factors I pointed to would tend to mean that the relative rank for the US is in reality better than listed, IMO, but the wide bounds of substitution adjustment makes it very hard to be confident in the results.
I don’t see that supported by the data, since other countries aren’t broken out by race.
Also, how biased is the sample? What percentage of Americans take the test? And Europeans?
PISA website.
In all countries, schools are sampled, attempting to control for some variables. Around 5000 students, 160 schools in the US. Note that schools may/may not choose to participate, and the same with students. PISA has minimum standards for acceptance as sampled selection, in an attempt to avoid the obvious bias that countries would have an interest to produce in their samples.
http://nces.ed.gov/surveys/pisa/faq.asp
Given the optional sampling, and obvious incentives, I’m skeptical that this is particularly accurate, and an apples to apples comparison.
Also, homeschoolers, one of the strongest demographics for ethnic and intact family reasons, are likely missed by this sample, skewing results in the US down. They appear to be about 4% of the total population, disproportionately white and in two parent households.
They control for schools opting not to participate; see the section under substitute schools.
It’s standard research ethics that minors (and their guardians) be given the option to refuse to participate in a study.
Now I’m confused. The original inference Sailer drew from PISA was that American students outperform their racial group in other countries. You’re claiming the study will be biased against white Americans. If anything you should be annoyed at Sailer for trying to support his racial performance narrative using a study that didn’t really focus on it.
Regarding homeschooling in particular, it’d be nearly impossible to develop an international study on the same scale of PISA (which you already want to reject as too small) merely because homeschooling isn’t prevalent in most of the PISA-participating countries.
I’m aware that they tried to control, but the bounds are large and open to exploitation by the countries who so choose to do so.
And the homeschool issue should strengthen his conclusion. I was just noting a factor he hadn’t controlled for.
Both factors I pointed to would tend to mean that the relative rank for the US is in reality better than listed, IMO, but the wide bounds of substitution adjustment makes it very hard to be confident in the results.