An interesting idea would be to feed people the scientific data that ancient or medieval scientists had and see whether they reproduced all the incorrect but (given the limited knowledge) plausible theories that were invented.
This would work especially well on the vast numbers of people in our society who don’t know any science anyway.
In fact just finding some sufficiently obscure area of current science would suffice. There’s so much of it… How much of contemporary paleontology or inorganic chemistry could I re-invent?
I once succeeded in deriving the solution to the cubic and quartic equations invented by Tartaglia in the 1500′s. It took me all day and almost the whole night, plus some tidying up the next day. I think it made a good exercise in rationality, and was harder than I thought it would be.
Related to this, many elite universities face the same kind of problem that we have here: they have to put a lot of effort into sorting out the good wrote learners from the creative thinkers in their crop of prospective students. To do this they have various entrance exams, plus interviews.
One way to test the effectiveness of rationality arts would be to offer rationality training to 17⁄18 year olds who were soon to be interviewed by these top universities. If our group did significantly better than one would have predicted based on their grades, then we would have a quantifiable and objective signal of effectiveness.
I strongly second the idea of using real science as a test. Jeffreyssai wouldn’t be satisfied with feeding his students—even the beginners—artificial puzzles all day. Artificial puzzles are shallow.
It wouldn’t even have to be historical science. Science is still young enough that there’s a lot of low-hanging fruit. I don’t think we have a shortage of scientific questions which are genuinely unanswered, but can be recognized as answerable in a moderate amount of time by a beginner or intermediate student.
Just to mention in passing, when I read your particular example, my immediate thought was “right, I’d fail right away”. Someone who sucks at math would probably find it very difficult to derive those solutions. Yet, I don’t think that means they couldn’t be rational. You’d have to take into account their personal skills and affinity in the scientific domain you’re testing, and adjust for that.
An interesting idea would be to feed people the scientific data that ancient or medieval scientists had and see whether they reproduced all the incorrect but (given the limited knowledge) plausible theories that were invented.
This would work especially well on the vast numbers of people in our society who don’t know any science anyway.
In fact just finding some sufficiently obscure area of current science would suffice. There’s so much of it… How much of contemporary paleontology or inorganic chemistry could I re-invent?
I once succeeded in deriving the solution to the cubic and quartic equations invented by Tartaglia in the 1500′s. It took me all day and almost the whole night, plus some tidying up the next day. I think it made a good exercise in rationality, and was harder than I thought it would be.
Related to this, many elite universities face the same kind of problem that we have here: they have to put a lot of effort into sorting out the good wrote learners from the creative thinkers in their crop of prospective students. To do this they have various entrance exams, plus interviews.
One way to test the effectiveness of rationality arts would be to offer rationality training to 17⁄18 year olds who were soon to be interviewed by these top universities. If our group did significantly better than one would have predicted based on their grades, then we would have a quantifiable and objective signal of effectiveness.
I strongly second the idea of using real science as a test. Jeffreyssai wouldn’t be satisfied with feeding his students—even the beginners—artificial puzzles all day. Artificial puzzles are shallow.
It wouldn’t even have to be historical science. Science is still young enough that there’s a lot of low-hanging fruit. I don’t think we have a shortage of scientific questions which are genuinely unanswered, but can be recognized as answerable in a moderate amount of time by a beginner or intermediate student.
Just to mention in passing, when I read your particular example, my immediate thought was “right, I’d fail right away”. Someone who sucks at math would probably find it very difficult to derive those solutions. Yet, I don’t think that means they couldn’t be rational. You’d have to take into account their personal skills and affinity in the scientific domain you’re testing, and adjust for that.
All the plausible but incorrect theories? Why? Guessing other people’s inferior answers doesn’t demonstrate rationality. It demonstrates empathy.