I would like to see tests for rationality discussed, criticized and designed. The equivalent of a IQ but for rationality. Just like for IQ scores it would not have to be perfect to be useful. Knowing a persons level of rationality is incredibly useful if it can be done in a reproducible and accurate way.
Absolutely. I’m particularly interested in ways of measuring group rationality, since a lot of the more important dysfunctions of thought show up only through group dynamics and since it’s hard to get anything really important done as a lone hacker or researcher.
I can think of some useful ways of measuring subvalues contributing to this value (even something as simple as paying attention at poker night goes a long way), but coming up with a workable general metric has eluded me so far.
I don’t know if it can be reliably measured—my impression is that it’s pretty damn hard, and that once you heard about most tests (the kind that are used to show irrationality in the first place, i.e. those already discussed on less wrong), you won’t fall for them but that doesn’t say much about how less likely you are for thinking right in “real life” situations where your brain isn’t primed by “this is a test of rationality”.
Some sub-components can be reliably measured—such as calibration. Any others?
My thought on hearing the proposal was that it would be impractically difficult, but on further consideration I suspect it would be much easier than creating a reliable test for intelligence. With proper effort, we should at least be able to beat the standard set by IQ tests
I got through unscathed, but I suspect most atheists would. I don’t think that test is very good at discerning rationality beyond the “knows about logic” and “doesn’t believe in God”.
FWIW, I started it with “God Exists” as true and also got through it unscathed. But you’re right, it doesn’t seem to try very hard presenting atheists with pitfalls and traps. I was at least expecting some kind of foul trickery about the Big Bang.
I would expect it to be extremely hard. I do not limit the scope of these tests to simple paper question and answer. I would be ok if an MRI was needed, for the teste not to know the test was about rationality, if it required a group of well trained actors to fool the teste, or a virtual simulation(within todays technology). Of course tests that can be used/verified immediately by users are preferable.
In the end the techniques required may be prohibitive, but at least it would make known what techniques/technologies to watch/promote for the future.
I would like to see tests for rationality discussed, criticized and designed. The equivalent of a IQ but for rationality. Just like for IQ scores it would not have to be perfect to be useful. Knowing a persons level of rationality is incredibly useful if it can be done in a reproducible and accurate way.
Absolutely. I’m particularly interested in ways of measuring group rationality, since a lot of the more important dysfunctions of thought show up only through group dynamics and since it’s hard to get anything really important done as a lone hacker or researcher.
I can think of some useful ways of measuring subvalues contributing to this value (even something as simple as paying attention at poker night goes a long way), but coming up with a workable general metric has eluded me so far.
I don’t know if it can be reliably measured—my impression is that it’s pretty damn hard, and that once you heard about most tests (the kind that are used to show irrationality in the first place, i.e. those already discussed on less wrong), you won’t fall for them but that doesn’t say much about how less likely you are for thinking right in “real life” situations where your brain isn’t primed by “this is a test of rationality”.
Some sub-components can be reliably measured—such as calibration. Any others?
My thought on hearing the proposal was that it would be impractically difficult, but on further consideration I suspect it would be much easier than creating a reliable test for intelligence. With proper effort, we should at least be able to beat the standard set by IQ tests
Battleground God
I got through unscathed, but I suspect most atheists would. I don’t think that test is very good at discerning rationality beyond the “knows about logic” and “doesn’t believe in God”.
FWIW, I started it with “God Exists” as true and also got through it unscathed. But you’re right, it doesn’t seem to try very hard presenting atheists with pitfalls and traps. I was at least expecting some kind of foul trickery about the Big Bang.
I would expect it to be extremely hard. I do not limit the scope of these tests to simple paper question and answer. I would be ok if an MRI was needed, for the teste not to know the test was about rationality, if it required a group of well trained actors to fool the teste, or a virtual simulation(within todays technology). Of course tests that can be used/verified immediately by users are preferable.
In the end the techniques required may be prohibitive, but at least it would make known what techniques/technologies to watch/promote for the future.