More precisely, what is rationality’s method for scoring matches? If you don’t have that, you have no way to know whether the flying guillotine is any good, or whether you’re even getting better at what you’re doing within your own school.
To me, the score worth caring about most, is how many of your own irrational beliefs, biases, conditioned responses, etc., you can identify and root out… using verifiable criteria for their removal… as opposed to simply being able to tell that it would be a good idea to think differently about something. (Which is why I consider Eliezer “formidable”, as opposed to merely “smart”: his writing shows evidence of having done a fair amount of this kind of work.)
Unfortunately, this sort of measurement is no good for scoring matches, unless the participants set out to prove at the beginning that they were more wrong than their opponent, at the beginning of the “match”!
But then, neither is any other sort of competitive measurement any good, as far I can see. If you use popularity, then you are subject to rhetorical effects, apparent intelligence, status, and other biasing factors. If you use some sort of reality-based contest, the result needn’t necessarily correlate with rationality or thinking skills in general. And if you present a puzzle to be solved, how will you judge the solution, unless you’re at least as “formidable” as the competitors?
More precisely, what is rationality’s method for scoring matches? If you don’t have that, you have no way to know whether the flying guillotine is any good, or whether you’re even getting better at what you’re doing within your own school.
To me, the score worth caring about most, is how many of your own irrational beliefs, biases, conditioned responses, etc., you can identify and root out… using verifiable criteria for their removal… as opposed to simply being able to tell that it would be a good idea to think differently about something. (Which is why I consider Eliezer “formidable”, as opposed to merely “smart”: his writing shows evidence of having done a fair amount of this kind of work.)
Unfortunately, this sort of measurement is no good for scoring matches, unless the participants set out to prove at the beginning that they were more wrong than their opponent, at the beginning of the “match”!
But then, neither is any other sort of competitive measurement any good, as far I can see. If you use popularity, then you are subject to rhetorical effects, apparent intelligence, status, and other biasing factors. If you use some sort of reality-based contest, the result needn’t necessarily correlate with rationality or thinking skills in general. And if you present a puzzle to be solved, how will you judge the solution, unless you’re at least as “formidable” as the competitors?
Any system of measurement is subject to Goodhart’s Law. This is really rough when you’re trying to engage with reality.