I like your second argument. But to be honest, there is a giant grey area between “non-replicable” and “fraudulent”. It is hard to draw the line between, “Intellectually dishonest but didn’t mean to deceive” and “fraudulent”. And even if you could define the line, we lack the data to identify what falls on either side.
It is worth reading Time to assume that health research is fraudulent until proven otherwise? I believe that this case is an exception to Betteridge’s law—I think that the answer is yes. Given the extraordinary efforts that the editor of Anaesthesia needed to catch some fraud, I doubt that most journals do it. And because I believe that, I’m inclined to a prior that says that non-replicability suggests at least even odds of fraud.
As a sanity check, high profile examples like the former President of Stanford demonstrate that fraudulent research is accepted in top journals, leading to prestigious positions. See also the case of Dr. Francesca Gino, formerly of Harvard.
And, finally, back to the line between intellectual dishonesty and fraud. I’m inclined to say that they amount to the same thing in practice, and we should treat them similarly. And the combined bucket is a pretty big problem.
Here is a good example. The Cargo Cult Science speech happened around 50 years ago. Psychologists have objected ever since to being called a pseudoscience by many physicists. But it took 40 years before they finally did what Feynman told them to, and tried replicating their results. They generally have not acknowledged Feynman’s point, nor have the started fixing the other problems that Feynman talked about.
Given that, how much faith should we put in psychology?
I like your second argument. But to be honest, there is a giant grey area between “non-replicable” and “fraudulent”. It is hard to draw the line between, “Intellectually dishonest but didn’t mean to deceive” and “fraudulent”. And even if you could define the line, we lack the data to identify what falls on either side.
It is worth reading Time to assume that health research is fraudulent until proven otherwise? I believe that this case is an exception to Betteridge’s law—I think that the answer is yes. Given the extraordinary efforts that the editor of Anaesthesia needed to catch some fraud, I doubt that most journals do it. And because I believe that, I’m inclined to a prior that says that non-replicability suggests at least even odds of fraud.
As a sanity check, high profile examples like the former President of Stanford demonstrate that fraudulent research is accepted in top journals, leading to prestigious positions. See also the case of Dr. Francesca Gino, formerly of Harvard.
And, finally, back to the line between intellectual dishonesty and fraud. I’m inclined to say that they amount to the same thing in practice, and we should treat them similarly. And the combined bucket is a pretty big problem.
Here is a good example. The Cargo Cult Science speech happened around 50 years ago. Psychologists have objected ever since to being called a pseudoscience by many physicists. But it took 40 years before they finally did what Feynman told them to, and tried replicating their results. They generally have not acknowledged Feynman’s point, nor have the started fixing the other problems that Feynman talked about.
Given that, how much faith should we put in psychology?