Climate science: how it matters for understanding forecasting, materials I’ve read or plan to read, sources of potential bias
As part of a review of forecasting, I’ve been looking at weather and climate forecasting (I wrote one post on weather forecasting and another on the different time horizons for weather and climate forecasting).
Climate forecasting is turning out to be a fairly tricky topic to look into, partly because of the inherent complexity of the task, and partly because of the politicization surrounding Anthropogenic Global Warming (AGW).
Due to the complexity and the potential for bias, I decided to disclose what materials I’ve read and my potential sources of bias.
Why am I looking at climate forecasting?
Climate forecasting, and the debate surrounding what’ll happen to the climate and how human choices today can shape it, is one of the biggest examples of a long-range forecasting effort that has attracted widespread attention, both in terms of the science and the policy and political implications. Understanding how it was done can give insights into the ability of humans to make forecasts about the long-run future (on the decadal or centennial timescale) in the face of considerable uncertainty, and use those forecasts to drive decisions today. This would be relevant for other long-range forecasting problems, such as (possibly) friendly AI. Note though that my focus isn’t driven by finding parallels with any other specific forecasting problem, such as friendly AI.
The sorts of questions I hope to answer by the end of this inquiry
The following are questions to which I hope to state relatively clear answers by the end:
How good are we at climate forecasting?
How good are we at knowing how good we are at climate forecasting? Are the forecasts appropriately calibrated, or do they tend to be overconfident or underconfident?
Are climate forecasters using the best tools available to them from other domains (such as statistics, econometrics, forecasting, weather forecasting)? Are they using best practices in their efforts?
What is the level of evidence regarding Anthropogenic Global Warming (AGW) and to what extent have the people generally deferred to as experts correctly weighed the evidence?
The following are questions to which I may not obtain clear answers, but I’ll be looking for and reporting information on them because they influence the answers to the preceding questions:
Given that climate forecasts, and the AGW hypothesis in particular, have been considered a basis for significant collective action (such as restricting emissions, or subsidies to alternative energy sources), there are obviously big political stakes in the outcome of the science. Oil and coal companies, particularly if they don’t anticipate being easily able to diversify, stand to lose from policy measures, while nuclear, solar. and wind energy companies might gain. To what extent have these vested interests influenced the science?
More generally, to what extent have people’s beliefs about the possible political consequences about specific outcomes affected the science in ways that are not epistemically justified? For instance, do people who are more risk-averse tend to exaggerate the harms, so that they can convince a less risk-averse public to take action? Do people who view restrictions on carbon dioxide emissions as economically disastrous tend to downplay the scientific evidence for AGW in order to minimize the probability of emissions reduction legislation?
Sources
Courses or full-fledged reviews
David Archer’s global warming Coursera course (Archer is a climate scientist specializing in ocean-related stuff at the University of Chicago, and one of the bloggers at RealClimate).
Books about climate change aimed at a popular audience
Six Degrees: Our Future on a Hotter Planet by Mark Lynas (Amazon, Wikipedia)): I only read the chapters about warming up to 3 degrees Celsius. The focus of my inquiry is the climate forecasting itself, not so much the consequences of it, but I did want to get a handle on what sorts of consequences people expect.
Coming Climate Crisis? Consider the Past, Beware the Big Fix by Claire J. Parkinson (Amazon): I have read Chapters 1 and 5 so far, and intend to read/skim other chapters when writing about relevant material.
Books about specific controversies surrounding climate change
The Hockey Stick Illusion: Climategate and the Corruption of Science by Andrew Montford (Amazon, Wikipedia): I read almost the whole book (skipping some pages of the last chapter). Despite the subtitle, the book is not about Climategate but rather about the debate surrounding the hockey stick graph. The graph is actually quite peripheral to the central debates of climate science, but the debate surrounding it provides important insight into the sociology of climate science and the IPCC process.
The Climate Files: The Battle for the Truth about Global Warming by Fred Pearce (Amazon): I read the whole book.
Book chapters
Chapter 12 of Nate Silver’s The Signal and the Noise. This chapter is about climate science, and specifically about anthropogenic global warming. The book also has a chapter on weather forecasting that I read and used in an earlier post.
IPCC reports
I read a large part of Chapter 8 of the IPCC 4th Assessment Report Working Group I on climate change models and their evaluation, and skimmed the corresponding chapter in the IPCC 5th Assessment Report Working Group I.
I haven’t read the other IPCC Working Group I chapters yet, but intend to do so where relevant.
I don’t think the other Working Groups of the IPCC are too relevant for my purposes. Also, I’ve heard that the quality of reports in the other Working Groups leaves a lot to be desired. But I might refer to those if I need to understand more about the policy implications.
Blogs and websites
I reference here only the blogs and websites I’ve identified as places to check out, rather than ones where I chanced upon an isolated blog post by link-traipsing or searching the web.
Skeptical Science (website, Wikipedia): Unlike what the name suggests, it is not run by global warming skeptics but rather by people who seek to debunk global warming skepticism. I used this website mainly to understand both the standard arguments offered against the Anthropogenic Global Warming (AGW) hypothesis and the common mainstream rebuttals to these arguments. I found it to be a reasonably comprehensive compendium of arguments and rebuttals.
Watts Up With That? (WUWT) (website, Wikipedia) run by Anthony Watts (Wikipedia): I used this website extensively to understand non-mainstream and skeptical perspectives on climate science, as well as some aspects of the source of rancor and skepticism expressed by outside-the-establishment bloggers. As a general rule, whenever looking up a topic, I searched for it on WUWT. I found WUWT to have fairly thorough and comprehensive coverage and the individual posts to be quite long and detailed, but not all posts should be treated as reliable or on par with a published article. Each post should be evaluated at the object level.
Climate Audit (CA) (website, Wikipedia), run by Stephen McIntyre (Wikipedia): Although this too is labeled a skeptic site, it has much more limited scope. While WUWT covers any and all climate science-related topics and features guest posts from all sorts of people, CA is more focused on the modeling and statistical methods used in papers. As the name suggests, the purpose is more like an auditor than somebody attempting to sell a competing theory. I’ve used this to understand some of the controversies surrounding measurement, and get a sense of the politics and dynamics of disputes.
RealClimate (website, Wikipedia): This was the web’s first climate blog. In fact, it has been described in The Climate Files and The Hockey Stick Illusion as a way for climate scientists to regain control of the public debate in the face of all the Internet discussion among skeptics critiquing their papers. That being said, I didn’t find the posts there very useful for understanding the issues involved. Part of it might be the combative tone used, part of it was the low frequency of posting, and part of it was the lack of mathematical detail accompanying many of the posts.
Judith Curry’s blog (website, Wikipedia on Curry): Curry is an interesting people because she identifies as a mainstream scientist but also engages with, and highly respects, the work of skeptic websites such as WUWT and CA.
Papers
I read many papers from a diverse array of sources. I arrived at most papers either by clicking links on one of the blogs or websites mentioned above, or using Google or Google Scholar searches for specific topics. Any paper that I use as input to my opinion in a specific post will be explicitly linked in that post.
Potential for bias and inaccuracy
My political views lean libertarian, and although I don’t think this affects my view of the plausibility of climate theories directly, it does affect the intellectual environment I operate in (less deferential to the mainstream consensus). I don’t think this was an issue, since the list of sources I used were mostly derived using Google Search and Wikipedia as starting points, rather than my libertarian friends. But it might have affected me. Some people have also argued that since libertarians oppose heavy-handed government intervention, they have an incentive to not believe in anthropogenic global warming since it presents an “inconvenient truth” for their position.
I don’t know much about the subject. The above reading list is hardly enough to train myself in climate science. How might my lack of knowledge bias me? It might make me too sensitive to presentation. In particular, this may lead me to take positions espoused in skeptic blogs such as WUWT and CA more seriously: the authors combine (what seems to be) a careful examination of the data with a desire to get at the truth of whatever empirical issue they are investigating, and they share in considerable detail their thought process. In contrast, the Real Climate blog posts are more like announcements than investigations I feel part of. But this does not mean that WUWT or CA is more reliable, of course: the climate scientists blogging at Real Climate are more busy writing up stuff for publication than sharing it on blogs. Much as I might prefer the blogging culture to the paper-writing culture, I should avoid using this as an important input in my evaluation of the legitimacy of specific scientific claims.
Looking for suggestions
As always, I’m happy to hear suggestions. In particular, I am interested in suggestions on these fronts:
Additional sources I should refer to
Cautions or caveats for reading or interpreting the sources already on my list (or perhaps a suggestion to read some of the already listed sources more thoroughly)
Other sources of bias I might have that I missed
Potential ways to correct for my bias
- The insularity critique of climate science by (9 Jul 2014 1:17 UTC; 14 points)
- Tentative tips for people engaged in an exercise that involves some form of prediction or forecasting by (30 Jul 2014 5:24 UTC; 14 points)
- Domains of forecasting by (9 Jul 2014 13:45 UTC; 11 points)
- [QUESTION]: What are your views on climate change, and how did you form them? by (8 Jul 2014 14:52 UTC; 9 points)
- Time series forecasting for global temperature: an outside view of climate forecasting by (7 Jul 2014 16:25 UTC; 5 points)
- Carbon dioxide, climate sensitivity, feedbacks, and the historical record: a cursory examination of the Anthropogenic Global Warming (AGW) hypothesis by (8 Jul 2014 1:58 UTC; 4 points)
That seems likely, given what we know of human biases, though I think that people would normally do that unconsciously rather than intentionally . This paper by Alhakami and Slovic shows that people who think, e.g. that nuclear power is risky don’t think it’s beneficial, while those who think it’s beneficial think it’s risky (they explain this by reference to the halo effect):
Yes, I would expect this to be likely too. But I’m interested in collecting specific object-level evidence of this being the case. (I’ve already collected some information, but I’m still processing and checking it).
-- I think you meant.
John Baez’s Azimuth project/community might be of interest to you.
http://johncarlosbaez.wordpress.com/
I’ve noticed that you’ve listed a lot of secondary sources (books, blogs, IPCC summaries) but not primary sources (published papers by scientists in peer-reviewed journals). Is there a reason for this e.g. that you do not have access to the primary sources, or find them indigestible?
If you do need to rely on secondary sources, I’d suggest to focus on books and blogs whose authors are also producing the primary sources. Of the blogs you mention, I believe that Real Climate and Skeptical Science are largely authored by working climate scientists, whereas the others are not.
Of course a number of the blogs convey the message that climate science as a whole is utterly partisan and biased, so any output of climate scientists through secondary sources and summaries is untrustworthy. If you can’t analyse the underlying primary evidence, and do not assign negligible prior probability to such a mass scientific conspiracy (or mass scientific error) then it is hard to refute that mindset. But you still have to ask who has the greater incentives here: is it really poorly paid scientists pushing a conspiracy or collective fantasy to get a bit more funding, or is it highly paid lobbyists, firms and commentators defending a trillion dollar industry, one which would be doomed by serious action on climate change?
As I mentioned in the post:
I can’t include a list of papers right now because the list of papers will itself be determined in real time during my inquiries, but I will link to the ones I reference at each stage of research.
This needs to be unpacked.
Scientists are driven by a range of motives including research prestige and ego (those who have made statements in the past want to have those statements vindicated), the desire to impress and influence peers, etc. Getting more funding is part of the status game. These incentives can distort their findings, not necessarily forever but for long enough (just like the climate system, the system of scientific discovery is not in equilibrium; there are lags).
More importantly, simple cognitive and human biases can get in the way of proper analysis. One of the points I raise is that it’s possible, based on Armstrong and Green’s critique (I’m still investigating this) that climate scientists appear not to have consulted people in other relevant areas of expertise (specifically, forecasting and statistics). People in these areas of expertise have come up with a lot of relevant and counter-intuitive findings about how to go about this sort of tricky data analysis.
Of the sources I list, which ones do you think is written or funded by people who fit this description?
Judith Curry is also a working climate scientist.
Stephen McIntyre isn’t a working climate scientist, but his criticism of Mann’s statistical errors (which aren’t necessarily relevant to the main arguments for AGW) have been acknowledged as essentially correct. I also took a reasonably detailed look at the specifics of the argument, and although I can’t have very high confidence, I’m inclined to believe that McIntyre was right. He seems to be sufficiently rigorous in his work and a sufficiently strong skeptic that his critiques are worth reading. Moreover, he rarely claims more confidence than is warranted: he isn’t publishing his own theories of climate change, he sees it as his job mainly to catch bad statistics.
Watts Up With That? seems to have poorer quality control in general, and often publishes opinion-type posts, but many of those opinion posts are guest posts by working climate scientists (albeit not necessarily high-profile ones). i wouldn’t assign high credence to the blog.
Did you have a look at these responses? Or at Mann’s book on the subject?
There are a number of points here, but the most compelling is that the statistical criticisms were simply irrelevant. Contrary to McIntyre and McKitrick’s claims, the differences in principal component methodology make no difference to the proxy reconstructions. And the hockey stick graph has since been replicated dozens of times using multiple different proxies and methods, by multiple authors.
On funding, it can be difficult to trace: see this article in Scientific American and the original paper plus the list of at least 91 climate counter-movement organisations, page 4, which have an annual income of over $900 million. A number of these organisations are known to have received funding by companies like Exxon and Koch Industries, though the recent trend appears to be more opaque funding through foundations and trusts.
On your particular sources, Climate Audit is on that list; also, from his Wikipedia bio it appears that Steve McIntyre was the founder and president of Northwest Explorations inc until it was taken over by CGX Resources to form the oil and gas exploration company CGX Energy Inc. He was a “strategic advisor” to CGX Energy at the time of his critique of the “hockey stick” in 2003.
Anthony Watts has received funding from the Heartland Institute#Affiliation_with_Heartland_Institute) which is also on the list. He claims it was not for the WUWT blog, and he approached them rather than the other way round.
Judith Curry has admitted to receiving “some funding from the fossil fuel industry” DeSmogBlog quoting Scientific American though again she claims no correlation with her views.