Bem believes that an informal Bayesian filter (extraordinary claims require extraordinary evidence)
Bem does not believe that most researchers really follow extraordinary claims require extraordinary evidence. He believes that many of the relevant researches won’t be convinced regardles of what evidence is provided.
He might be wrong about that belief but saying that he believes that most researchers would be convinced be reasonable data misunderstands Bem.
methodology is flawed enough that its not much evidence of anything
Not much evidence and no evidence are two different things. If he believes it’s evidence and you don’t he’s right. It might not be much evidence but it’s evidence in the bayesian sense.
If you debate with him in person and pretend it’s no evidence he will continue to say it’s evidence and be right. That will prevent the discussion to come to the questions that actually matter of how strong the evidence happens to be.
The replication attempts that failed further reinforce this belief.
At university we did a failed attempt to replicate PCR. It really made the postdoc who was running the experiement ashamed that she couldn’t get it right and that it failed for some reason unknown to her. In no way does this concludes that PCR doesn’t work.
As far as replication goes Bem also seems to think that there were successful replication attempts:
What Wiseman never tells people is in Ritchie, Wiseman and French is that his online registry where he asked everyone to register, first of all he provided a deadline date. I don’t know of any serious researcher working on their own stuff who is going to drop everything and immediately do a replication… anyway, he and Ritchie and French published these three studies. Well, they knew that there were three other studies that had been submitted and completed and two of the three showed statistically significant results replicating my results.
If you have a very strange effect that you don’t understand and can’t pin down having 2 of 6 replication attempts be successful does not really prove that there no effect.
If something can go wrong and a method like PCR that’s done millions of times fails to replicated without knowledgeable people knowing why, failing to replicate a very new effect doesn’t mean much. Trying to pin down the difference between the 2 successful and the 4 failed replication attempts might be in order. At least that where I would focus my attention when I’m not attached to the outcome. It may very well turn out that there no real effect in the end but there seems to be more than nothing.
From the same interview of Bem I linked to above (but by the moderator):
How ironic that would be, since one of the strategies of the debunkers has long been to psychologize the phenomena and say, “We don’t need to study the phenomena. We need to study these weird people who report and believe these weird things.” Wouldn’t it be ironic if it turns out we need to look at the beliefs and psychology of the experimenters and why they don’t believe and why they don’t get these effects.
Again that not that much different from the way Sokal sees the literature department.
Bem does not believe that most researchers really follow extraordinary claims require extraordinary evidence. He believes that many of the relevant researches won’t be convinced regardles of what evidence is provided.
He might be wrong about that belief but saying that he believes that most researchers would be convinced be reasonable data misunderstands Bem.
Not much evidence and no evidence are two different things. If he believes it’s evidence and you don’t he’s right. It might not be much evidence but it’s evidence in the bayesian sense.
If you debate with him in person and pretend it’s no evidence he will continue to say it’s evidence and be right. That will prevent the discussion to come to the questions that actually matter of how strong the evidence happens to be.
At university we did a failed attempt to replicate PCR. It really made the postdoc who was running the experiement ashamed that she couldn’t get it right and that it failed for some reason unknown to her. In no way does this concludes that PCR doesn’t work.
As far as replication goes Bem also seems to think that there were successful replication attempts:
If you have a very strange effect that you don’t understand and can’t pin down having 2 of 6 replication attempts be successful does not really prove that there no effect. If something can go wrong and a method like PCR that’s done millions of times fails to replicated without knowledgeable people knowing why, failing to replicate a very new effect doesn’t mean much. Trying to pin down the difference between the 2 successful and the 4 failed replication attempts might be in order. At least that where I would focus my attention when I’m not attached to the outcome. It may very well turn out that there no real effect in the end but there seems to be more than nothing.
From the same interview of Bem I linked to above (but by the moderator):
Again that not that much different from the way Sokal sees the literature department.