Any new education method will show increases in student test scores if people believe it results in increases in student test scores, because only interested parents will sign up for that method.
For example, Freakonomics tells the story of high school students in Chicago who participated in a lottery for the chance to switch schools. The students who were reassigned to new schools were more likely to graduate; but the students who applied for the lottery but lost did just as well. The explanation given is that the students (or parents) who care about education will attempt to switch to a better school, but the “better” school won’t confer an advantage.
Cullen, Jacob, and Levitt. “The Impact of School Choice on Student Outcomes: An Analysis of the Chicago Public Schools”. J. Public Econ. 200?.
Cullen, Jacob, and Levitt. “The Effect of School Choice on Student Outcomes: Evidence from Randomized Lotteries”. National Bureau of Economic Research working paper, 2003.
There was a study done regarding the cause and effect of employee relationships and how it affected job performance that gave as a result that employees performed better simply because of the attention given them, rather than the validity of any of the techniques introduced.
If anyone can provide a resource for that study, I’ll vote you up because I am having trouble finding it.
If I remember correctly it is used in:
O’Hair, Friedrich, Dixon 2008 Strategic Communication: In Business and the Professions, Pearson
It’s called The Hawthorne Effect. At least in the Wikipedia article, no one considers the possibility that mere variation (less boredom) improves performance.
I was thinking of that exact example with regards to the posts discussion on hypothetical Montessori schools.
The filtering effect still wouldn’t vanish—instead of filtering FOR the most engaged parents it would filter AGAINST the least engaged (I.E. all the parents that cared would put their kids into Montessori, the parents who didn’t care would just put their kids into whatever school was the most convenient for them)
You are right. Unfortunately, “any parent who cared at all” is still a significant selection.
Also there are different kinds of “care”. For example a parent may spend their time choosing the right school, and even pay more money for the school… and yet completely ignore what their children are doing at the school. The filter of Montessori schools may work on both these levels—require both selection and later parent participation. I am not familiar with details of Montessori schools.
A data point: I taught at a school where student grades were not written on paper, but only on a school website. Parents received a password they could use to see their child’s grades. During the year 1⁄3 passwords were never used. That means that parents of 1⁄3 children either did not care what grades their child has or at least did not care to verify what their child reported. It was a school that required special tests for a child to be accepted, and for many children it was inconveniently far from their home—so parents did care about choosing the right school, but then 1⁄3 of them stopped caring about what happened in the school.
I guess this could be explained by signalling. Choosing your child’s school is a one-time activity that signals that you are a great parent. Reading your child’s grades online is invisible (unless someone curious looks into logs later), therefore useless for signalling.
For example, Freakonomics tells the story of high school students in Chicago who participated in a lottery for the chance to switch schools. The students who were reassigned to new schools were more likely to graduate; but the students who applied for the lottery but lost did just as well. The explanation given is that the students (or parents) who care about education will attempt to switch to a better school, but the “better” school won’t confer an advantage.
Cullen, Jacob, and Levitt. “The Impact of School Choice on Student Outcomes: An Analysis of the Chicago Public Schools”. J. Public Econ. 200?.
Cullen, Jacob, and Levitt. “The Effect of School Choice on Student Outcomes: Evidence from Randomized Lotteries”. National Bureau of Economic Research working paper, 2003.
The following link lends credence to this line of thought:
http://www.management.wharton.upenn.edu/grant/Grant_JAP2008b_TaskSignificance.pdf
A note:
There was a study done regarding the cause and effect of employee relationships and how it affected job performance that gave as a result that employees performed better simply because of the attention given them, rather than the validity of any of the techniques introduced.
If anyone can provide a resource for that study, I’ll vote you up because I am having trouble finding it.
If I remember correctly it is used in: O’Hair, Friedrich, Dixon 2008 Strategic Communication: In Business and the Professions, Pearson
It’s called The Hawthorne Effect. At least in the Wikipedia article, no one considers the possibility that mere variation (less boredom) improves performance.
Thanks NancyLebovitz, that’s the one.
| [...]no one considers the possibility that mere variation (less boredom) improves performance.
The reverse possibility may also be true, more boredom decreases performance and may also cause health problems.
R.U.R.)
I was thinking of that exact example with regards to the posts discussion on hypothetical Montessori schools.
The filtering effect still wouldn’t vanish—instead of filtering FOR the most engaged parents it would filter AGAINST the least engaged (I.E. all the parents that cared would put their kids into Montessori, the parents who didn’t care would just put their kids into whatever school was the most convenient for them)
You are right. Unfortunately, “any parent who cared at all” is still a significant selection.
Also there are different kinds of “care”. For example a parent may spend their time choosing the right school, and even pay more money for the school… and yet completely ignore what their children are doing at the school. The filter of Montessori schools may work on both these levels—require both selection and later parent participation. I am not familiar with details of Montessori schools.
A data point: I taught at a school where student grades were not written on paper, but only on a school website. Parents received a password they could use to see their child’s grades. During the year 1⁄3 passwords were never used. That means that parents of 1⁄3 children either did not care what grades their child has or at least did not care to verify what their child reported. It was a school that required special tests for a child to be accepted, and for many children it was inconveniently far from their home—so parents did care about choosing the right school, but then 1⁄3 of them stopped caring about what happened in the school.
I guess this could be explained by signalling. Choosing your child’s school is a one-time activity that signals that you are a great parent. Reading your child’s grades online is invisible (unless someone curious looks into logs later), therefore useless for signalling.