Although I agree that a theory born of empirical evidence is better than one born of introspection, I think it is kind of dangerous to introspect, develop a theory, and then when you’re posting it on Less Wrong look for some evidence to support it so that you can say it’s empirical. It risks reducing The Procurement of Evidence to a ritual.
The matter would be different if there happened to be a study about this exact topic, or if there had been some study that had inspired him to come up with this theory. But “come up with theory, find supporting evidence” seems dangerous to me.
Oh, good point. I think of “come up with theory, think about what it implies, look for evidence one way or the other” as the ideal, but the difficulty is that confirming information is more salient in my memory than disconfirming.
On the other hand, filtered evidence is still evidence, and a lack of outside evidence can be a sign that there’s no good confirming evidence. (Or, in this case, just a sign that the poster is new around here.)
Although I agree that a theory born of empirical evidence is better than one born of introspection, I think it is kind of dangerous to introspect, develop a theory, and then when you’re posting it on Less Wrong look for some evidence to support it so that you can say it’s empirical. It risks reducing The Procurement of Evidence to a ritual.
See, the problem is, he could probably tie the evidence about choking into his theory. But if he had the opposite theory, he could probably tie studies like the ones showing mental practice can improve sports performance and the one showing that problem-solving areas of the brain are highly active when we daydream in to support that. That means that the fact that he can find a tangentially related study doesn’t really make it more likely that the post is true. It’d just make us feel all nice and empirical
The matter would be different if there happened to be a study about this exact topic, or if there had been some study that had inspired him to come up with this theory. But “come up with theory, find supporting evidence” seems dangerous to me.
Isn’t the answer simply that one shouldn’t misinterpret what it means for evidence to be supporting?
Oh, good point. I think of “come up with theory, think about what it implies, look for evidence one way or the other” as the ideal, but the difficulty is that confirming information is more salient in my memory than disconfirming.
On the other hand, filtered evidence is still evidence, and a lack of outside evidence can be a sign that there’s no good confirming evidence. (Or, in this case, just a sign that the poster is new around here.)