Experimental and Organizational tests seem to be the most important test types here; if the students and methods are able to show they’re capable, and are measurably better than the students of another craft, then their school is obviously doing something better than other schools anyway, no Reputational test needed. So I’ll concentrate on those.
What do we need for an experimental test? We need a way of comparing the strengths of students and ideas, to see which are stronger. The problem here is that there’s not really a standard unit of rationality. If you want to measure something’s volume, you can put it in a water bath and measure how many mL it displaces. If you want to measure someone’s rationality… you’re a bit out of luck.
I’m not well versed enough yet in cognitive sciences to propose a unit of raw intelligence/rationality measurement, and a way of at least estimating it. Until such a metric is apparant, I think we can make do with comparative testing. Take two students and have them perform some test of rationality that returns less rational, more rational, or equally rational as a rough comparison of the two. Perform it on an entire school, and you can rank each student. Perform it between similarly ranked students in two schools, and you can determine which school is better. Roughly. A test like this could also potentially serve as an organizational test.
What tests would I propose as an experiment? How about something like having the students competitively build a weirdtopia? (http://lesswrong.com/lw/xm/building_weirdtopia/)
You could have a panel of randomly selected scifi fans read one of the two weirdtopias (don’t compare them side by side, we’re trying to get their honest opinion about one of the stories, not their comparison of the two) and rate 1-10 how much they’d like to live in that weirdtopia. The student with a higher voted paper is more rational, and if the stories are about equally weighted, we have two roughly equal weirdtopias.
That… doesn’t test every facet of rationality I know. However, using tests as a way of comparing two students is something that a lot more tests could be adapted to, without necessarily having to make a measurable yardstick of rationality. Just need to figure out which aspect of rationality you want to test, look at papers and stories that display this aspect, have the two students write a similar paper using their own skills, and compare the two.
Experimental and Organizational tests seem to be the most important test types here; if the students and methods are able to show they’re capable, and are measurably better than the students of another craft, then their school is obviously doing something better than other schools anyway, no Reputational test needed. So I’ll concentrate on those.
What do we need for an experimental test? We need a way of comparing the strengths of students and ideas, to see which are stronger. The problem here is that there’s not really a standard unit of rationality. If you want to measure something’s volume, you can put it in a water bath and measure how many mL it displaces. If you want to measure someone’s rationality… you’re a bit out of luck.
I’m not well versed enough yet in cognitive sciences to propose a unit of raw intelligence/rationality measurement, and a way of at least estimating it. Until such a metric is apparant, I think we can make do with comparative testing. Take two students and have them perform some test of rationality that returns less rational, more rational, or equally rational as a rough comparison of the two. Perform it on an entire school, and you can rank each student. Perform it between similarly ranked students in two schools, and you can determine which school is better. Roughly. A test like this could also potentially serve as an organizational test.
What tests would I propose as an experiment? How about something like having the students competitively build a weirdtopia? (http://lesswrong.com/lw/xm/building_weirdtopia/) You could have a panel of randomly selected scifi fans read one of the two weirdtopias (don’t compare them side by side, we’re trying to get their honest opinion about one of the stories, not their comparison of the two) and rate 1-10 how much they’d like to live in that weirdtopia. The student with a higher voted paper is more rational, and if the stories are about equally weighted, we have two roughly equal weirdtopias.
That… doesn’t test every facet of rationality I know. However, using tests as a way of comparing two students is something that a lot more tests could be adapted to, without necessarily having to make a measurable yardstick of rationality. Just need to figure out which aspect of rationality you want to test, look at papers and stories that display this aspect, have the two students write a similar paper using their own skills, and compare the two.