Deworming a movement
Over the last few days I’ve been reviewing the evidence for EA charity recommendations. Based on my personal experience alone, the community seems to be comprehensively inept, poor at marketing, extremely insular, methodologically unsophisticated but meticulous, transparent and well-intentioned. I currently hold the belief that EA movement building does more harm than good and that is requires significant rebranding and shifts in its informal leadership or to die out before it damages the reputation of the rationalist community and our capacity to cooperate with communities that share mutual interests.
It’s one thing to be ineffective and know it. It’s another thing to be ineffective and not know it. It’s yet another thing to be ineffective, not know it, yet champion effectiveness and make a claim to moral superiority.
In case you missed the memo deworming is controversial, GiveWell doesn’t engage with the meat of the debate, and my investigations of the EA community’s spaces suggests that it’s not at all known. I’ve even briefly posted about it elsewhere on LessWrong to see if there was unspoken knowledge about it, but it seems not. Given that it’s the hot topic in mainstream development studies and related academic communities, I’m aghast at how irresponsive ‘we’ are.
What’s actionable for us here. If you’re looking for a high reliability effective altruism prospect, do not donate to SCI or Evidence Action. And by extension, do not donate to EA organisations to donate to these groups, including GiveWell. I am assuming you will use those funds more wisely instead, say buying healthier food for yourself.
For who don’t to review the links for a more comprehensive analyses from Cochrane and GiveWell, here is one summary of the debate recommended in the Cochrane article:
Last month there was another battle in an ongoing dispute between economists and epidemiologists over the merits of mass deworming. In brief, economists claim there is clear evidence that cheap deworming interventions have large effects on welfare via increased education and ultimately job opportunities. It’s a best buy development intervention. Epidemiologists claim that although worms are widespread and can cause illnesses sometimes, the evidence of important links to health is weak and knock-on effects of deworming to education seem implausible. As stated by Garner “the belief that deworming will impact substantially on economic development seems delusional when you look at the results of reliable controlled trials.”
Aside: Framing this debate as one between economists and epidemiologists captures some of the dynamic of what has unfortunately been called the “worm wars” but it is a caricature. The dispute is not just between economists and epidemiologists. For an earlier round of this see this discussion here, involving health scientists on both sides. Note also that the WHO advocates deworming campaigns.
So. Deworming: good for educational outcomes or not?
On their side, epidemiologists point to 45 studies that are jointly analyzed in Cochrane reports. Among these they see few high quality studies on school attendance in particular, with a recent report concluding that they “do not know if there is an effect on school attendance (very low quality evidence).” Indeed they also see surprisingly few health benefits. One randomized control trial included one million Indian students and found little evidence of impact on health outcomes. Much bigger than all other trials combined; such results raise questions for them about the possibility of strong downstream effects. Economists question the relevance of this result and other studies in the Cochrane review.
On their side, the chief weapon in the economists’ arsenal has for some time been a paper from 2004 on a study of deworming in West Kenya by Ted Miguel and Michael Kremer, two leading development economists that have had an enormous impact on the quality of research in their field. In this paper, Miguel and Kremer (henceforth MK) claimed to show strong effects of deworming on school attendance not just for kids in treated schools but also for the kids in untreated schools nearby. More recently a set of new papers focusing on longer term impacts, some building on this study, have been added to this arsenal. In addition, on their side, economists have a few things that do not depend on the evidence at all: determination, sway, and the moral high ground. After all, who could be against deworming kids?
Additional criticisms of GiveWelL charities: http://lesswrong.com/lw/mo0/open_thread_aug_24_aug_30/cp8h
The kind of work I think EA’s should be focussing on http://lesswrong.com/lw/mld/genosets/cnys AND
The problem with MIRI: http://lesswrong.com/lw/cr7/proposal_for_open_problems_in_friendly_ai/cm2j
You could stand to be more explicit in your reasoning. At the moment it seems to go like this:
One paper that found big benefits from deworming has recently been subject to criticism, which criticism has in turn been criticized, etc.
GiveWell posted some discussion of the debate that you think doesn’t engage with the meat of the issue.
It seems to me to engage with quite a lot, and you don’t say what it is you think they aren’t engaging with.
Therefore GiveWell are inept.
Therefore, giving to GiveWell’s recommended charities is worse than, and I quote, buying healthier food for yourself.
It seems to me that no part of this argument makes much sense. There are intelligent experts on both sides of the “worm wars”; GiveWell’s reasoning doesn’t seem obviously crazy to me (their support for deworming was never entirely based on M&K’s findings; many of those findings hold up under the currently-debated reanalysis; the EA case for deworming was always that although the effectiveness of deworming is highly uncertain it’s really really cheap and the current estimates of its cost-effectiveness would need to be too high by an order of magnitude or more for it to stop being better than, e.g. giving money directly to the beneficiaries—which, you may note, is an intervention that GiveWell are also recommending, so it’s not as if they aren’t discounting deworming somewhat on account of the uncertainty); none of this seems to justify the extremely strong words you use.
If you have a more detailed argument that actually gets from the available evidence to “the EA movement is hopelessly messed up”, let’s hear it. That would be interesting and important. But for the moment I’m afraid I’ve got you in the same mental pigeonhole as others who’ve come along to LW and said “I’ve found some nits to pick with something EAs tend to approve of; therefore we should give up the whole idea of charity and concentrate on benefitting ourselves”.
My arguments weren’t meant to be logically water tight to lead to an undeniable position refuting a particular position, they were meant to start a discussion.
Why is it messed up? Lack of competition. Why is there so little competition in the cause prioritization space? Why aren’t there more organisations saying EA’s, donate to this best cause? Some may say it’s an economy of scale thing, but I feel like it has lead to EA orgs becoming quite methodologically lazy. While there are lots of incredible strengths, we also have weaknesses and despite open solicitation for criticism, whenever I for one criticise aspects of the movement I get shot down without my actual arguments getting shot down.
I hate to keep being the resident cost-effectiveness skeptic/denialist and getting tonnes of downvotes for it (but someone has to be the Red Team in the EA movement), but I wanted to clarify and admit that I misunderstood much of the controversy around deworming and that is it contained to: 1) the impact of MASS deworming v.s. individual deworming), which is relevant to deciding whether charitable deworming (like Deworm the world or SCI) rather than user fee style deworming (private businesses, charity distortion free, free markets) is appropriate and (2) whether deworming is actually that much a good public health intervention. I like effective altruism cause I have a very low tolerance for the idea that I might be donating to wasteful causes, and I feel like this is something that would concern many EA’s and we ought to investigate it. I wish an informal committee of some sort was convened to routinely challenge or dogmas cause I feel we have been stagnating and are increasingly becoming complacent, naïve (see GiveWell’s underwhelming responses to the Cochrane review disfavourful of deworming) and thus increasingly unsexy. Ps. I feel like altruism is way to contentious a word for a movement that could be less about ‘ethics’ and other vague philosophical vocabulary and more about economy and charity, which describes what we actually do.
Because most people don’t value everyone’s welfare equally, and that’s a basic assumption of EA. It’s like asking why there aren’t more organizations selling lollipops containing insects.
You hedge your claim with phrases like ‘based on my personal experience’, “seems’, ‘hold the belief’, but many people will read over these as the point you make in the first sentence is quite strong. Extraordinary claims demand extraordinary evidence but you present only one extended point—and one that is controversial. This might give the impression that you have only this one. I suggest that you reference more of the evidence you have reviewed.
It may give the impression I have only one point, but that’s all I’m trying to do. I’m trying to point out that the approach to EA at the moment tends to be very cultish, at least in my local area. It’s not hard to find the relevant arguments furthering the position I’m hinting at, but they are already well developed by academic experts, just not presented within EA spaces online. I’m a shit writer, but I am knowledgable. Perhaps someone else can better communicate what I’m suggesting.
Deworming is controversial because it’s an area where the people doing the work where open to high quality scientific analysis of the effects of their work. GiveWell does not only have the mission of getting money to charities that at the moment are efficient but it also has the mission of incentivising charities to be transparent.
That alone shouldn’t drive recommendation but it’s valuable to keep it in mind.
Why haven’t you linked to GiveWell’s report on the evidence for Deworming? Which of the claims that GiveWell makes in that document do you find to be clearly wrong?
That’s actually completely inaccurate. Deworming and the worm wars is used as a classic example of science held back by poor disclosure. Why haven’t I linked to GiveWell’s report on deworming? Because anyone interested in this area would probably have been exposed to it and I’m not talking to the hundred people interested in EA vaguely, but the 10 or so who look deep into the evidence and methodology. Obliteration by incorporation.
I’ve heard of the controversy. I think it was mentioned in a link post on slatestarcodex, and obviously on GiveWell’s blog.
I find it stylistically strange to have a long list of negative adjectives end with two positive ones (transparent and well-intentioned are good things, right?) without any explanation. Wouldn’t one say something like “These things suck:...., but on the good side there is also ….”?
More importantly, you do not explain why “EA movement building does more harm than good”.
I understand you to mean “EA movement building does more harm than good, because the EA movement does more harm than good” (stop me right there if I miss understood you). Why though?
As I understood it, no one argues that de-worming does more harm than good. The argument is only that it is ineffective, not harmful. If you want to make some argument that de-worming takes away resources that should be better spend, you have to actually make that argument.
Could you explain what’s so bad about GiveWell’s reaction, particularly the blog post you linked? Not just where you disagree with their analysis, but how that post is evidence that GiveWell is more harmful than beneficial.
Finally, even if the EA-movement is wrong about de-worming, there are other interventions that EA tends to support. Your post isn’t very convincing right now because it doesn’t mention that fact at all. Do you think that all interventions popular among EAs are on as shaky a ground as de-worming (or worse)?
I think the “but” was the transition, and that “meticulous” was also intended positively.
I was under the impression that specialists worried that mass deworming leads to resistance, by standard evolutionary logic, and so argue that the deworming initiatives are committing a long-term harm for nonexistent short-term gains.
That’s one hypothesis, but there isn’t much compelling evidence for it. Although there is good reason to believe the evidence for mass deworming as the among the ‘best’ interventions in shakey, there is not good reasons to believe it’s a ‘bad’ intervention either.
Style isn’t a big priority to me unless is compromises understandabiltiy by those interested enough to summount readability challenges. I assume that from their ownwards worthy concepts will prolliferate into the broader population.
It’s not mentioned anywhere on SSC AFAIK. I wrote this post because it’s absent in the rationality sphere. GiveWell’s treatise is quite pathetic honestly, but I won’t be posting a critique of it because (1) it would shoot down a high quality organisation and may do more harm then good and (2) it would be an effortful undertaking that I would prefer to publish under a more reputable pseudonym.
That’s very obvious though. Deworming consistutures roughly half the suggested charities of most EA orgs, so I think it’s fair to say methodological issues reflect on the whole movement.