For an especially blatant example of this, consider their bizarre claim that peer review is somehow an essential part of science—whereas in reality, as anyone with even a cursory familiarity with the history of science knows, peer review is a bureaucratic innovation that has been widely imposed only in the decades since WW2, and science had functioned perfectly well for centuries before that.
Peer review existed in some forms well before that. The major journals like the various journals of the Royal Society often had pieces vetted by the editors rather than independent referees, but the point of independent review was not uncommon. Also, if one looks at the RW article on peer review, one sees that they list some of the more serious problems with the system.
The only bit of their section on peer review that seems really problematic is this:
If an idea has not been published in a single peer review journal, it is safe to say it is not science.
Whereas in physics and math and some other areas, many results come out in preprint form well before they are formally published (and in math their are a few examples of fairly important results that exist only in preprint form). Would you object to this sentence instead saying ” If an idea has not been published in a single peer review journal, it is a warning sign that the idea may not be science?”
Most of the rest of the article seems ok to me, although others of your points may be valid, especially in regard to the low status nature of most of their examples (although in many locations alt med is pretty high status, especially in the American left-wing which undermines your claim somewhat.)
Peer review existed in some forms well before that. The major journals like the various journals of the Royal Society often had pieces vetted by the editors rather than independent referees, but the point of independent review was not uncommon.
That is true, of course, but the sorts of editorial review that were standard before the mid-20th century were different in some very important ways. When Max Planck accepted Einsten’s annus mirabilis papers, the decision was under his personal responsibility, and he was putting his own reputation on the line. This is a very different procedure from today’s reliance on anonymous reviewers and impersonal committees. It’s also very different from a setup where officially designated “peers” (i.e. insiders of an single exclusive group) serve as bureaucratic judges of what is valid science about some issue and what isn’t. (See this old comment in which I elaborated on some important aspects of these differences.)
Also, if one looks at the RW article on peer review, one sees that they list some of the more serious problems with the system.
Notice that the “pseudoscience” article confuses peer review and replication. This clearly indicates a writer who has a very poor understanding of the present institutions of official science, but is nevertheless driven by a strong desire to mount ideological crusades on their behalf. Furthermore, the linked “peer review” article informs us that “peer review is a key part of the scientific method.” This is just plain false: whether or not a given work follows the scientific method depends only on the way it’s conceived and executed, and not at all on whether it’s been given some kind of bureaucratic imprimatur. Robinson Crusoe would be perfectly capable of using the scientific method, limited only by his own knowledge and the resources available on the island.
Furthermore, I don’t think the “peer review” article goes anywhere near a comprehensive critique of the system. It doesn’t even discuss the most obvious problems that struck me the same moment I was assigned peer review work for the first time some years ago. But that’s a topic for a separate long discussion.
Would you object to this sentence instead saying ” If an idea has not been published in a single peer review journal, it is a warning sign that the idea may not be science?”
It depends on the field in question. In some fields (basically the hardest of the hard sciences), it is indeed a pretty good heuristic. At the other extreme, in some other fields the situation is so bad that publication in a peer-reviewed journal strongly indicates that even if the work has some valid insight, it’s buried in a quagmire of bias and bad epistemology—whereas on the other hand, perfectly valid insight is often made outside of any official accredited institutions.
In yet other cases, valid science can be done entirely unofficially by amateurs, and this isn’t so rare when it comes to topics that are of little or no academic interest.
Most of the rest of the article seems ok to me, although others of your points may be valid, especially in regard to the low status nature of most of their examples (although in many locations alt med is pretty high status, especially in the American left-wing which undermines your claim somewhat.)
I didn’t claim that RW falls for left-wing nonsense in general. If the left-wing nonsense in question is opposed by the mainstream academia, or if it’s too far left to be embraced by the respectable left-center intellectual and media institutions, RW will also be opposed to it. (Although, again, its criticism will end up biased insofar as these institutions are biased towards the issue.)
The real problem is what happens if pseudoscience is in fact done by the official and prestigious academia and with all the official bureaucratic trappings of “science” in place. A truly correct and informative treatment of “pseudoscience” should give us some idea of how to recognize this situation. Yet the RW writers seem incapable of even conceiving such a possibility, and their entire output is driven by the desire to equate “pseudoscience” with non-academic contrarian positions. (Or, in case of the example of The Bell Curve, with work whose conclusions are unacceptable to the ideological left-center, even though they are more or less in line with the beliefs of a substantial number of academics specializing in the issues it deals with, and the work has some academic affiliation.) Now, you may disagree whether it’s fair to label such bias as “ideological”—although I think it’s an appropriate description—but I don’t see on what grounds you would disagree with the diagnosis of the problem itself.
Peer review existed in some forms well before that. The major journals like the various journals of the Royal Society often had pieces vetted by the editors rather than independent referees, but the point of independent review was not uncommon. Also, if one looks at the RW article on peer review, one sees that they list some of the more serious problems with the system.
The only bit of their section on peer review that seems really problematic is this:
Whereas in physics and math and some other areas, many results come out in preprint form well before they are formally published (and in math their are a few examples of fairly important results that exist only in preprint form). Would you object to this sentence instead saying ” If an idea has not been published in a single peer review journal, it is a warning sign that the idea may not be science?”
Most of the rest of the article seems ok to me, although others of your points may be valid, especially in regard to the low status nature of most of their examples (although in many locations alt med is pretty high status, especially in the American left-wing which undermines your claim somewhat.)
That is true, of course, but the sorts of editorial review that were standard before the mid-20th century were different in some very important ways. When Max Planck accepted Einsten’s annus mirabilis papers, the decision was under his personal responsibility, and he was putting his own reputation on the line. This is a very different procedure from today’s reliance on anonymous reviewers and impersonal committees. It’s also very different from a setup where officially designated “peers” (i.e. insiders of an single exclusive group) serve as bureaucratic judges of what is valid science about some issue and what isn’t. (See this old comment in which I elaborated on some important aspects of these differences.)
Notice that the “pseudoscience” article confuses peer review and replication. This clearly indicates a writer who has a very poor understanding of the present institutions of official science, but is nevertheless driven by a strong desire to mount ideological crusades on their behalf. Furthermore, the linked “peer review” article informs us that “peer review is a key part of the scientific method.” This is just plain false: whether or not a given work follows the scientific method depends only on the way it’s conceived and executed, and not at all on whether it’s been given some kind of bureaucratic imprimatur. Robinson Crusoe would be perfectly capable of using the scientific method, limited only by his own knowledge and the resources available on the island.
Furthermore, I don’t think the “peer review” article goes anywhere near a comprehensive critique of the system. It doesn’t even discuss the most obvious problems that struck me the same moment I was assigned peer review work for the first time some years ago. But that’s a topic for a separate long discussion.
It depends on the field in question. In some fields (basically the hardest of the hard sciences), it is indeed a pretty good heuristic. At the other extreme, in some other fields the situation is so bad that publication in a peer-reviewed journal strongly indicates that even if the work has some valid insight, it’s buried in a quagmire of bias and bad epistemology—whereas on the other hand, perfectly valid insight is often made outside of any official accredited institutions.
In yet other cases, valid science can be done entirely unofficially by amateurs, and this isn’t so rare when it comes to topics that are of little or no academic interest.
I didn’t claim that RW falls for left-wing nonsense in general. If the left-wing nonsense in question is opposed by the mainstream academia, or if it’s too far left to be embraced by the respectable left-center intellectual and media institutions, RW will also be opposed to it. (Although, again, its criticism will end up biased insofar as these institutions are biased towards the issue.)
The real problem is what happens if pseudoscience is in fact done by the official and prestigious academia and with all the official bureaucratic trappings of “science” in place. A truly correct and informative treatment of “pseudoscience” should give us some idea of how to recognize this situation. Yet the RW writers seem incapable of even conceiving such a possibility, and their entire output is driven by the desire to equate “pseudoscience” with non-academic contrarian positions. (Or, in case of the example of The Bell Curve, with work whose conclusions are unacceptable to the ideological left-center, even though they are more or less in line with the beliefs of a substantial number of academics specializing in the issues it deals with, and the work has some academic affiliation.) Now, you may disagree whether it’s fair to label such bias as “ideological”—although I think it’s an appropriate description—but I don’t see on what grounds you would disagree with the diagnosis of the problem itself.