Rationality skill: Recognize rationality skill in others.
There is a strong tendency for people to use the heuristic, “P(Person X is right) = # of times person X has been observed to be right / # of statements person X has made”, or worse, “(# statements by X - # of times X has admitted to making a mistake) / # statements by X”.
This encourages people who want to be respected to do 4 bad things:
Make many pronouncements on things that are obviously right, perhaps in a manner suggesting they are controversial claims.
Avoid saying anything unless they are certain they are correct.
Avoid saying anything concrete enough to possibly be proven wrong.
Never, ever admit to having made a mistake.
Unfortunately, these methods are extremely effective.
I think the key here is qualification—Robert Morris avoided being wrong by not stating things unqualified unless he was sure of them, whereas the failure mode for rationalists is not expressing an idea at all unless fairly sure of it.
We want ideas to be shared before they’re well-supported, because discussion is generally the best way for them to find support (or disproof) - we just need to signal the uncertainty when we introduce an idea.
It’s much like what I’ve been taught in analytical chemistry—every number has a stated uncertainty associated with it.
It can easily be both; and, while not speaking unless very confident may not be terribly harmful by itself, the perfectionism that it’s an example of is a very characteristic failure mode of rationalists.
To some extent, those are also signs of someone low on the social hierarchy (or someone who feels low on the social hierarchy), and a punitive culture that punishes individuals for brainstorming or otherwise being visibly wrong.
Rationality skill: Recognize rationality skill in others.
There is a strong tendency for people to use the heuristic, “P(Person X is right) = # of times person X has been observed to be right / # of statements person X has made”, or worse, “(# statements by X - # of times X has admitted to making a mistake) / # statements by X”.
This encourages people who want to be respected to do 4 bad things:
Make many pronouncements on things that are obviously right, perhaps in a manner suggesting they are controversial claims.
Avoid saying anything unless they are certain they are correct.
Avoid saying anything concrete enough to possibly be proven wrong.
Never, ever admit to having made a mistake.
Unfortunately, these methods are extremely effective.
I don’t think this is a pernicious behavior at all. I suspect that this is actually a sign of rationality.
http://www.paulgraham.com/heroes.html (See Robert Morris)
I think the key here is qualification—Robert Morris avoided being wrong by not stating things unqualified unless he was sure of them, whereas the failure mode for rationalists is not expressing an idea at all unless fairly sure of it.
We want ideas to be shared before they’re well-supported, because discussion is generally the best way for them to find support (or disproof) - we just need to signal the uncertainty when we introduce an idea.
It’s much like what I’ve been taught in analytical chemistry—every number has a stated uncertainty associated with it.
It can easily be both; and, while not speaking unless very confident may not be terribly harmful by itself, the perfectionism that it’s an example of is a very characteristic failure mode of rationalists.
To some extent, those are also signs of someone low on the social hierarchy (or someone who feels low on the social hierarchy), and a punitive culture that punishes individuals for brainstorming or otherwise being visibly wrong.