Let’s go back and look at the source article one more time: “PubMed references more than 25 million articles relating primarily to biomedical research published since the 1940s. A comprehensive search of the PubMed database in May 2012 identified 2,047 retracted articles, with the earliest retracted article published in 1973 and retracted in 1977.”
So over 99.99% of articles aren’t retracted. Lets say the retracted ones are a tip of the iceberg and the real situation is ten times worse. That makes it 99.9% accurate.
Aside from the sensationalism, these results are a stunning and unequivocal endorsement that the scientific system works.
Aside from the sensationalism, these results are a stunning and unequivocal endorsement that the scientific system works.
2047 articles retracted (for any reason) out of 25 million = 0.008%. (Edit: I mistyped the figure as 5 million instead of 25 million, but the percentage was correct.) 21.3% of that was retracted due to error, i.e. 0.0017% of all published papers later admitted an error that made the paper worthless.
That looks like massive underreporting. I cannot believe that there weren’t several orders of magnitude more retraction-worthy cases, which were not retracted. In fact, this is so massive that I would characterize it as unequivocal endorsement that the scientific system is broken, because nobody is willing to admit to their (honest, non fraudulent) errors, and few publish failed reproductions.
Analyzing the few papers that were retracted will surely have big confounding factors, because it’s a preselected subset of papers that is already abnormal in some way (why were they retracted? Did a third party discover the fraud? Is this selection for the least competent fraudsters?)
I don’t have good data on the subject, and I’m not well-calibrated. But my expectation of the rate of severe errors, made by well-meaning people in a complex endeavor, is for at least 1% retractions (3.5 orders of magnitude above the current 0.0017%). That would still be less than one lifetime retraction per scientist on average.
And another thought: if scientists are really so thorough as to achieve a very low rate of major errors, they are probably overspending. It would be more efficient (fewer false negatives in self-vetting and less time spent self-vetting) to be bolder in publishing and rely more on vetting and reproduction by others.
Let’s go back and look at the source article one more time: “PubMed references more than 25 million articles relating primarily to biomedical research published since the 1940s. A comprehensive search of the PubMed database in May 2012 identified 2,047 retracted articles, with the earliest retracted article published in 1973 and retracted in 1977.”
So over 99.99% of articles aren’t retracted. Lets say the retracted ones are a tip of the iceberg and the real situation is ten times worse. That makes it 99.9% accurate.
Aside from the sensationalism, these results are a stunning and unequivocal endorsement that the scientific system works.
2047 articles retracted (for any reason) out of 25 million = 0.008%. (Edit: I mistyped the figure as 5 million instead of 25 million, but the percentage was correct.) 21.3% of that was retracted due to error, i.e. 0.0017% of all published papers later admitted an error that made the paper worthless.
That looks like massive underreporting. I cannot believe that there weren’t several orders of magnitude more retraction-worthy cases, which were not retracted. In fact, this is so massive that I would characterize it as unequivocal endorsement that the scientific system is broken, because nobody is willing to admit to their (honest, non fraudulent) errors, and few publish failed reproductions.
Analyzing the few papers that were retracted will surely have big confounding factors, because it’s a preselected subset of papers that is already abnormal in some way (why were they retracted? Did a third party discover the fraud? Is this selection for the least competent fraudsters?)
What rate of retraction would provide evidence that the system is working?
I don’t have good data on the subject, and I’m not well-calibrated. But my expectation of the rate of severe errors, made by well-meaning people in a complex endeavor, is for at least 1% retractions (3.5 orders of magnitude above the current 0.0017%). That would still be less than one lifetime retraction per scientist on average.
And another thought: if scientists are really so thorough as to achieve a very low rate of major errors, they are probably overspending. It would be more efficient (fewer false negatives in self-vetting and less time spent self-vetting) to be bolder in publishing and rely more on vetting and reproduction by others.