You claim there are significant issues with the climate science process, but admit there are no journal articles criticizing the process. If you know enough to find faults with their science, why haven’t you yourself written an article on the matter?
Do you think there is something inherent in the culture of climatology science that introduces these anti-Bayesian biases? Why is climate science subject to this when other sciences are not?
Are you saying the field is systemically politically driven from the top down?
Have you followed the climategate email leak story at all? One of the more damning themes in the leaked emails is the discussion of ways to keep dissenting views out of the peer reviewed journals. One of the stronger arguments used against AGW skeptics was that there were not more papers supporting their claims in peer reviewed journals. Given the prevalence of this argument, clear evidence of efforts to keep ‘dissenting’ opinions out of the main peer reviewed journals is a big problem for the credibility of climate science. For example:
The group also did not approve of the American Geophysical Union (AGU) and its choices allowing opposing views to be heard. The group’s trade publication, Geophysical Research Letters (GRL) was targeted by Michael Mann as he wrote, “I’m not sure that GRL can be seen as an honest broker in these debates anymore.” He however acknowledged the publications importance saying, “We can’t afford to lose GRL.”
Mann seemed particularly concerned about ‘contrarian’ with the name Saiers, presumably James Saiers of the Yale School of Forestry & Environmental Studies. “Apparently, the contrarians now have an “in” with GRL. This guy Saiers has a prior connection w/ the University of Virginia Dept. of Environmental Sciences [where Saiers completed his PhD] that causes m some unease,” Mann wrote.
Tom Wigley, a senior scientist in the Climate and Global Dynamics Division at NCAR, felt though that they could deal with Saiers by getting him removed from the AGU. “If you think that Saiers is in the greenhouse skeptics camp, then, if we can find documentary evidence of this, we could go through official AGU channels to get him ousted.”
This was the danger of always criticising the skeptics for not publishing in the
“peer-reviewed literature”. Obviously, they found a solution to that—take over a journal!
So what do we do about this? I think we have to stop considering “Climate Research” as a legitimate peer-reviewed journal. Perhaps we should encourage our colleagues in the climate research community to no longer submit to, or cite papers in, this journal. We would also need to consider what we tell or request of our more reasonable colleagues who currently sit on the editorial board...
What, specifically, is “damning” about those quotes?
Suppose creationists took over a formerly respected biology journal. Wouldn’t you expect to find quotes like the above (with climate sceptics replaced by creationists) from the private correspondence of biologists?
AGW skeptics have often been challenged on the lack of peer reviewed papers in credible climate science journals supporting their arguments. Now it is quite possible that this is the case because skeptical papers have been rejected purely due to being bad science (as is the case with the lack of papers supporting the effectiveness of homeopathy in medical journals). However, the absence of papers from the key journals cannot be treated as independent evidence of the badness of the science if there is a concerted effort by AGW believers to keep such papers out of the journals.
It is legitimate to attack the science the AGW skeptics are doing. It is not legitimate to dismiss the science purely on the basis that they have not been published in peer reviewed journals if there is a concerted effort to keep them out of peer reviewed journals based on their conclusions rather than on their methods. Now I’m sure the AGW believers feel that they are rejecting bad science rather than rejecting conclusions they don’t like but emails like the above certainly make it appear that it is the conclusions as much as the methods that they are actually objecting to.
In my opinion the CRU emails mean that it no longer appears justified to ignore claims by AGW skeptics purely because they have not appeared in a peer reviewed journal. They may still be wrong but there is sufficient evidence of biased selection by the journals to not trust that journal publication is an unbiased signal of scientific quality.
Agreed. “No peer-reviewed publications” is not an argument that I’ve ever used or would use, even in advance of the CRU emails, because of course that is how academia works in general.
For the most part, I don’t think you’re quite answering my question.
You present two explanations for the lack of peer-reviewed articles that are sceptical of the scientific consensus on global warming. The first is that there is unjust suppression of such views. The second is that such scepticism is based on bad science. You say that you think the leaked emails support the first explanation, and that there is sufficient evidence of biased (I’m guessing “biased” means “unmerited by the quality of the science” here) selection by journals. What is that sufficient evidence? More specifically, how does the information conveyed by the leaked emails distinguish between the first and second scenarios?
Now I’m sure the AGW believers feel that they are rejecting bad science rather than rejecting conclusions they don’t like but emails like the above certainly make it appear that it is the conclusions as much as the methods that they are actually objecting to.
This addresses my questions, but I was asking for more specifics. Let A = “AGW sceptics are being suppressed from journals without proper evaluation of their science” and B = “AGW sceptics are being suppressed from journals because their science is unsound”. Let E be the information provided by the email leaks. How do you get to the conclusion that the likelihood ratio P(E|A)/P(E|B) is significantly above 1?
Personally I can’t see how the likelihood ratio would be anything but about 1, and it seems to me that those who act if the ratio is significantly greater than 1 are simply ignoring the estimation of P(E|B) because their prior for P(B) is small.
(EDIT: I originally wrote P(A|E) and P(B|E) instead P(E|A) and P(E|B). My text was still, apparently, clear enough that this wrong notation didn’t cause confusion. I’ve now fixed the notation.)
I do think the likelihood ratio is significantly above 1. This is based off reading some of the emails, documents and code comments in the leaks. Here’s a reasonable summary of the emails. It looks like dubious science to me. I find it hard to understand how anyone can claim otherwise unless they are ideologically motivated. If you genuinely can’t see it then I’m not really interested in arguing over minutiae so we’ll just have to leave it at that.
It seems to me that AGW skeptics made a variety of claims that AGW believers dismissed as paranoid: there was a conspiracy to keep skeptical papers out of the journals; there were efforts to damage the careers of climate scientists who didn’t ‘toe the party line’; there were dubious and possibly illegal efforts to keep the original data behind key papers out of the hands of skeptics despite FOI regulations. I did not see many AGW believers prior to the climategate emails saying “Yes, of course all of that happens, that’s just the way science operates in the real world”.
When the CRU leaks became public and substantiated all the ‘paranoid’ claims above, including proof of illegal destruction of emails and data to avoid FOI requests, I find it suspicious when people claim that it doesn’t change their opinions at all. The standard response seems to be “Oh yes, that’s just how science works in the real world. I already knew scientists routinely engage in this sort of behaviour and the degree of such behaviour revealed in the emails is exactly in line with my prior expectations so my probability estimates are unchanged”. That seems highly suspect to me and looks an awful lot like confirmation bias.
You’re still talking about how the e-mails fit into the scenario of fraudulent climate scientists, that is, P(E|A) by my notation. I specifically said that I feel P(E|B) is being ignored by those who claim the e-mails are evidence of misconduct. Your link, for example, mostly lists things like climatologists talking about discrediting journals that publish AGW-sceptical stuff, which is exactly what they would do if they, in good faith, thought that AGW-scepticism is based on quack science. Reading the e-mails and concluding that sceptical papers are being suppressed without merit seems like merely assuming the conclusion.
(Regarding the FOI requests, that might indeed be something that might reasonably set off alarms and significantly reduce P(E|B) - if you believe the sceptics’ commentaries accompanying the relevant quotes. But googling for “mcintyre foi harassment” and doing some reading gives a different story.)
My impression from reading the emails is that different standards are being applied to the AGW skeptics because of their conclusions rather than because of their methods. At the same time there is evidence of data massaging and dubious practices around their own methods in order to match their pre-conceived conclusions. The whole process does not look like the disinterested search for truth that is the scientific ideal.
My P(B|E) would be higher if I read emails that seemed to focus on methodological errors first rather than proceeding from the fact that a journal has published unwelcome conclusions to the proposal that the journal must be boycotted.
I think there’s too much attention paid to the emails, and not enough to all of the publicly available information about the exact same events. Maybe it’s because private communications seem like secret information that contain the hidden truth, or maybe it’s just a cascade effect where everyone focuses on the emails because everyone is focusing on the emails.
The second email that you quoted is in response to the publication of a skeptical article by Soon & Baliunas (2003) in the journal Climate Research which generated a big public controversy among climate scientists. Reactions to that publication include several editors of the journal resigning in protest (and releasing statements about why they resigned), the publisher of the journal writing a letter admitting that the article contained claims that weren’t supported by the evidence (pdf), and a scientific rebuttal to the article being published later that same year. I think that you get a better sense of what happened (and whether climate scientists were reacting to the methods or just the conclusions) by reading accounts written at the time than from the snippets of emails. And of course there’s Wikipedia.
FOI requests? Which ones? Those for proprietary data sets that they weren’t allowed at that time to release, or the FOI requests for information availalble from a public FTP site?
You claim there are significant issues with the climate science process, but admit there are no journal articles criticizing the process. If you know enough to find faults with their science, why haven’t you yourself written an article on the matter?
For the same reason I haven’t personally solved every injustice: a) time constraints, and b) others are currently raising awareness of this problem.
Do you think there is something inherent in the culture of climatology science that introduces these anti-Bayesian biases? Why is climate science subject to this when other sciences are not?
Other sciences are affected by anti-Bayesian biases, and this will be a tendency in proportion to the difficulty of finding solid evidence that your theory is wrong. Which is why I claim e.g. sociology and literature are mostly a waste of time.
Generally speaking, science is in some ways too strict and some ways not strict enough. Eliezer_Yudkowsky has actually pointed out before the general failure to appropriately teach rationality in the classroom, and so scientists in general aren’t aware of this problem.
Politics, of course, does play a part. When it’s not just about “who’s right” but about “who gets to control resources”, then the biases go into hyperdrive. People aren’t just pointing out problems with your research, they’re fighting for the other team! The goal is then about proving them wrong, not stopping to check whether your theory is correct in the first place. (“Ask whether, not why.”)
You claim there are significant issues with the climate science process, but admit there are no journal articles criticizing the process. If you know enough to find faults with their science, why haven’t you yourself written an article on the matter?
Do you think there is something inherent in the culture of climatology science that introduces these anti-Bayesian biases? Why is climate science subject to this when other sciences are not?
Are you saying the field is systemically politically driven from the top down?
Have you followed the climategate email leak story at all? One of the more damning themes in the leaked emails is the discussion of ways to keep dissenting views out of the peer reviewed journals. One of the stronger arguments used against AGW skeptics was that there were not more papers supporting their claims in peer reviewed journals. Given the prevalence of this argument, clear evidence of efforts to keep ‘dissenting’ opinions out of the main peer reviewed journals is a big problem for the credibility of climate science. For example:
And this comment is also rather damning:
What, specifically, is “damning” about those quotes?
Suppose creationists took over a formerly respected biology journal. Wouldn’t you expect to find quotes like the above (with climate sceptics replaced by creationists) from the private correspondence of biologists?
AGW skeptics have often been challenged on the lack of peer reviewed papers in credible climate science journals supporting their arguments. Now it is quite possible that this is the case because skeptical papers have been rejected purely due to being bad science (as is the case with the lack of papers supporting the effectiveness of homeopathy in medical journals). However, the absence of papers from the key journals cannot be treated as independent evidence of the badness of the science if there is a concerted effort by AGW believers to keep such papers out of the journals.
It is legitimate to attack the science the AGW skeptics are doing. It is not legitimate to dismiss the science purely on the basis that they have not been published in peer reviewed journals if there is a concerted effort to keep them out of peer reviewed journals based on their conclusions rather than on their methods. Now I’m sure the AGW believers feel that they are rejecting bad science rather than rejecting conclusions they don’t like but emails like the above certainly make it appear that it is the conclusions as much as the methods that they are actually objecting to.
In my opinion the CRU emails mean that it no longer appears justified to ignore claims by AGW skeptics purely because they have not appeared in a peer reviewed journal. They may still be wrong but there is sufficient evidence of biased selection by the journals to not trust that journal publication is an unbiased signal of scientific quality.
Agreed. “No peer-reviewed publications” is not an argument that I’ve ever used or would use, even in advance of the CRU emails, because of course that is how academia works in general.
For the most part, I don’t think you’re quite answering my question.
You present two explanations for the lack of peer-reviewed articles that are sceptical of the scientific consensus on global warming. The first is that there is unjust suppression of such views. The second is that such scepticism is based on bad science. You say that you think the leaked emails support the first explanation, and that there is sufficient evidence of biased (I’m guessing “biased” means “unmerited by the quality of the science” here) selection by journals. What is that sufficient evidence? More specifically, how does the information conveyed by the leaked emails distinguish between the first and second scenarios?
This addresses my questions, but I was asking for more specifics. Let A = “AGW sceptics are being suppressed from journals without proper evaluation of their science” and B = “AGW sceptics are being suppressed from journals because their science is unsound”. Let E be the information provided by the email leaks. How do you get to the conclusion that the likelihood ratio P(E|A)/P(E|B) is significantly above 1?
Personally I can’t see how the likelihood ratio would be anything but about 1, and it seems to me that those who act if the ratio is significantly greater than 1 are simply ignoring the estimation of P(E|B) because their prior for P(B) is small.
(EDIT: I originally wrote P(A|E) and P(B|E) instead P(E|A) and P(E|B). My text was still, apparently, clear enough that this wrong notation didn’t cause confusion. I’ve now fixed the notation.)
I do think the likelihood ratio is significantly above 1. This is based off reading some of the emails, documents and code comments in the leaks. Here’s a reasonable summary of the emails. It looks like dubious science to me. I find it hard to understand how anyone can claim otherwise unless they are ideologically motivated. If you genuinely can’t see it then I’m not really interested in arguing over minutiae so we’ll just have to leave it at that.
It seems to me that AGW skeptics made a variety of claims that AGW believers dismissed as paranoid: there was a conspiracy to keep skeptical papers out of the journals; there were efforts to damage the careers of climate scientists who didn’t ‘toe the party line’; there were dubious and possibly illegal efforts to keep the original data behind key papers out of the hands of skeptics despite FOI regulations. I did not see many AGW believers prior to the climategate emails saying “Yes, of course all of that happens, that’s just the way science operates in the real world”.
When the CRU leaks became public and substantiated all the ‘paranoid’ claims above, including proof of illegal destruction of emails and data to avoid FOI requests, I find it suspicious when people claim that it doesn’t change their opinions at all. The standard response seems to be “Oh yes, that’s just how science works in the real world. I already knew scientists routinely engage in this sort of behaviour and the degree of such behaviour revealed in the emails is exactly in line with my prior expectations so my probability estimates are unchanged”. That seems highly suspect to me and looks an awful lot like confirmation bias.
You’re still talking about how the e-mails fit into the scenario of fraudulent climate scientists, that is, P(E|A) by my notation. I specifically said that I feel P(E|B) is being ignored by those who claim the e-mails are evidence of misconduct. Your link, for example, mostly lists things like climatologists talking about discrediting journals that publish AGW-sceptical stuff, which is exactly what they would do if they, in good faith, thought that AGW-scepticism is based on quack science. Reading the e-mails and concluding that sceptical papers are being suppressed without merit seems like merely assuming the conclusion.
(Regarding the FOI requests, that might indeed be something that might reasonably set off alarms and significantly reduce P(E|B) - if you believe the sceptics’ commentaries accompanying the relevant quotes. But googling for “mcintyre foi harassment” and doing some reading gives a different story.)
(EDIT: Fixed notation, as in the parent.)
My impression from reading the emails is that different standards are being applied to the AGW skeptics because of their conclusions rather than because of their methods. At the same time there is evidence of data massaging and dubious practices around their own methods in order to match their pre-conceived conclusions. The whole process does not look like the disinterested search for truth that is the scientific ideal.
My P(B|E) would be higher if I read emails that seemed to focus on methodological errors first rather than proceeding from the fact that a journal has published unwelcome conclusions to the proposal that the journal must be boycotted.
I think there’s too much attention paid to the emails, and not enough to all of the publicly available information about the exact same events. Maybe it’s because private communications seem like secret information that contain the hidden truth, or maybe it’s just a cascade effect where everyone focuses on the emails because everyone is focusing on the emails.
The second email that you quoted is in response to the publication of a skeptical article by Soon & Baliunas (2003) in the journal Climate Research which generated a big public controversy among climate scientists. Reactions to that publication include several editors of the journal resigning in protest (and releasing statements about why they resigned), the publisher of the journal writing a letter admitting that the article contained claims that weren’t supported by the evidence (pdf), and a scientific rebuttal to the article being published later that same year. I think that you get a better sense of what happened (and whether climate scientists were reacting to the methods or just the conclusions) by reading accounts written at the time than from the snippets of emails. And of course there’s Wikipedia.
Would you expect to see evolutionary biologists discuss the methodological errors of creationist arguments in private correspondence?
(I don’t think this is the place for this, since I don’t think we’re getting anywhere.)
Upvoted for the parenthetical.
FOI requests? Which ones? Those for proprietary data sets that they weren’t allowed at that time to release, or the FOI requests for information availalble from a public FTP site?
Voted you up not for your particular assessment of P(A|E)/P(B|E) but for using this pattern of assessing evidence to guide the conversation.
For the same reason I haven’t personally solved every injustice: a) time constraints, and b) others are currently raising awareness of this problem.
Other sciences are affected by anti-Bayesian biases, and this will be a tendency in proportion to the difficulty of finding solid evidence that your theory is wrong. Which is why I claim e.g. sociology and literature are mostly a waste of time.
Generally speaking, science is in some ways too strict and some ways not strict enough. Eliezer_Yudkowsky has actually pointed out before the general failure to appropriately teach rationality in the classroom, and so scientists in general aren’t aware of this problem.
Politics, of course, does play a part. When it’s not just about “who’s right” but about “who gets to control resources”, then the biases go into hyperdrive. People aren’t just pointing out problems with your research, they’re fighting for the other team! The goal is then about proving them wrong, not stopping to check whether your theory is correct in the first place. (“Ask whether, not why.”)