[Link] Social interventions gone wrong
A piece I saw that Benjamin Todd adapted from THINK’s module on charity assessment. Some of you may recall the network’s recent launch.
Lots of social interventions end up doing more harm than good. Many more make no difference at all, and are just a waste of resources. At times, we’ve probably argued with friends about which interventions we’d like to see, and which we wouldn’t. But are we any good at judging what’s likely to work?
Here’s a cool bit of content adapted from THINK. Try and guess which of these eight programs made a difference, which had no effect, and which made things worse.
cipergoth said that it should be emphasised that this isn’t a trick question where the answer is they all worked or none did.
Round #1: Scared Straight
Program description: “In the 1970s, inmates serving life sentences at a New Jersey (USA) prison began a program to ‘scare’ or deter at‐risk or delinquent children from a future life of crime. The program, known as ‘Scared Straight’, featured as its main component an aggressive presentation by inmates to juveniles visiting the prison facility. The presentation depicted life in adult prisons, and often included exaggerated stories of rape and murder … The program received considerable and favorable media attention and was soon replicated in over 30 jurisdictions nationwide … Although the harsh and sometimes vulgar presentation in the earlier New Jersey version is the most famous, inmate presentations are now sometimes designed to be more educational than confrontational but with a similar crime prevention goal. Some of these programs featured interactive discussions between the inmates and juveniles, also referred to as ‘rap sessions.’(2)
Did the program decrease the rate of juvenile crime?
Round #2: Nurse‐Family Partnership
Program description: “The Nurse‐Family Partnership program provides nurse home visits to pregnant women with no previous live births, most of whom are i) low‐income, ii) unmarried, and iii) teenagers. The nurses visit the women approximately once per month during their pregnancy and the first two years of their children’s lives. The nurses teach i) positive health related behaviors, ii) competent care of children, and iii) maternal personal development (family planning, educational achievement, and participation in the workforce). The program costs approximately $12,500 per woman over the three years of visits (in 2010 dollars).”(6)
Did the program improve the quality of child care?
Round #3: Drug Abuse Resistance Education (DARE)
Program description: “DARE is a highly‐structured substance‐abuse prevention program taught by uniformed police officers … The program is typically provided over the course of 10‐20 weekly hour‐long sessions, during which the police officers use lectures, class discussion, role plays, and homework assignments to i) teach students about substance use and its effects, ii) teach students decision‐making and peer pressure resistance skills, and iii) boost students’ self‐esteem. Prior to teaching, the police officers take an 80‐hour training course on teaching techniques, classroom management, and the DARE curriculum … DARE costs approximately $130 (in 2004 dollars) per student and, as of 2001, was operating in 75% of American school districts.”(8)
Did the program decrease the rate of drug use?
Rounds #4 and #5: 21st Century Community Learning Centers
Program description: “21st Century Community Learning centers is a large ($1 billion per year) US Department of Education program which funds optional after‐school programs for elementary and middle school students in mostly high‐poverty schools. Key goals of the program are to i) provide students with a safe place after school, and ii) improve their academic performance. Recipients of program funds (ie, school districts and/or non‐profit educational/community organizations) are required to provide academically focused “extended learning activities” (e.g., instructional enrichment programs, tutoring, or homework assistance). Most centers also offer enrichment/recreational activities such as martial arts, sports, dance, art and/or music … (Elementary school) centers vary in the activities they offer and other key features, and thus comprise a range of after‐school interventions rather than a single intervention. In a typical center,
students may spend an hour doing homework and having a snack, an hour on additional academic activity (eg, a lesson or working in a computer lab), and an hour doing recreational or cultural activities;
the center’s staff are a mixture of certified teachers, instructional aides, and representatives of community youth organizations;
the center is open 4‐5 days per week for three hours after school, and serves approximately 85 students per day; and
the average student attends the center 2‐3 days per week.
Centers spend approximately $1,000 (in 2005 dollars) on each enrolled student per year.”(10)
Did the program increase the students’ academic achievement?
Did the program improve the behavioural problems at the schools?
Round #6: Even Start Family Literacy program
Program description: “The Even Start program is intended to ‘help break the cycle of poverty and illiteracy by improving the educational opportunities of the nation’s low‐income families by integrating early childhood education, adult literacy or adult basic education, and parenting education into a unified family literacy program’. In 2000‐2001, there were 855 Even Start projects serving 31,896 families … Even Start grantees had considerable flexibility in designing services to meet the needs of the low‐income families, but all were required to offer four services:
adult education to develop basic educational and literacy skills;
early childhood education services to provide developmentally appropriate services to help prepare children for school;
parenting education to help parents support the educational growth of their children; and
parent‐child literacy activities.”(13)
Did the program increase literacy?
Round #7: Big Brothers Big Sisters
Program description: “Big Brothers Big Sisters’ community‐based mentoring program matches youths aged 6‐18, predominantly from low‐income, single‐parent households, with adult volunteer mentors who are typically young (20‐34) and well‐educated (the majority are college graduates) … The mentor and youth typically meet for 2‐4 times per month for at least a year, and engage in activities of their choosing (e.g., studying, cooking, playing sports). The typical meeting lasts 3‐4 hours … For the first year, Big Brothers Big Sisters case workers maintain monthly contact with the mentor, as well as the youth and his or her parent, to insure a positive mentor‐youth match, and to help resolve any problems in the relationship. Mentors are encouraged to form a supportive friendship with the youths, as opposed to modifying the youth’s behavior or character… In 2008, Big Brothers Big Sisters served 255,000 youths and 470 agencies nationwide. The national average cost of making and supporting a match is approximately $1,300 in 2009 dollars.”(14)
Did the program decrease drug use and violent behavior?
Round #8: Top 16 Educational Software
Program description: “In the No Child Left Behind Act of 2002, Congress called for a rigorous study of the effectiveness of educational technology for improving student academic achievement … In fall 2003, developers and vendors of educational technology products responded to a public invitation and submitted products for possible inclusion in the national study. Mathematica Policy Research, Inc. staff selected 40 of the 160 submissions for further review by two panels of outside experts, one for reading products and one for math products … In January 2004, (the US Department of Education] considered the panel’s recommendations and selected 16 products for the study. In selecting products, (the US Department of Education) grouped them into four areas:
early reading (first grade),
reading comprehension (fourth grade),
pre‐algebra (sixth grade), and
algebra (ninth grade).
The products ranged widely in their instructional approaches and how long they had been in use. In general, however, the criteria weighted the selection towards products that had evidence of effectiveness from previous research, or, for newer products, evidence that their designs were based on approaches found to be effective by research. Twelve of the sixteen products had received awards or been nominated for awards (some as recently as 2006) by trade associations, media, teachers, or parents.”(15)
Did the program improve test scores?
Here are the answers!
Round #1: Scared Straight
Negative! Several randomized controlled trials have shown that Scared Straight had a negative effect. Going through Scared Straight made children more likely to commit crimes in the future (3). Fun fact: Scared Straight programs are still being run today (4), and people promote them as being effective, despite the fact that they are harmful (5).
Round #2: Nurse‐Family Partnership
Positive! Three randomized controlled trials have shown that the Nurse‐Family Partnership had a positive effect. The program led to a reduction in child abuse/neglect, child injuries (20‐50% reduction) and an improvement in cognitive/educational outcomes for children of mothers with low mental health/confidence/intelligence (e.g., 6 percentile point increase in grade 1‐6 in reading/math achievement) (7).
Round #3: Drug Abuse Resistance Education (DARE)
No effect!Two randomized controlled trials have shown that DARE did not have an effect on the rate of drug use among participants. The rate of drug use did not increase or decrease (9).
Round #4: 21st Century Community Learning Centers
No effect! A randomized controlled trial has shown that the 21st Century Community Learning Centers had no effect on participating students’ academic performance. Students who participated were neither helped nor harmed by the program.(11)
Round #5: 21st Century Community Learning Centers
Negative! A randomized controlled trial has shown that the 21st Century Community Learning Centers caused an increase in the behavioral problems of participating students (12).
Round #6: Even Start Family Literacy Program
No effect! A randomized controlled trial on a subset of Even Start programs found no evidence of an increase or decrease in literacy in parents or children (17).
Round #7: Big Brothers Big Sisters
Positive! A randomized controlled trial has shown that Big Brothers Big Sisters caused youths to be 46% less likely to have started using illegal drugs, 27% less likely to have started using alcohol, 32% less likely to have hit someone in the previous year and fewer days of skipping school during the past year (18).
Round #8: Top 16 Educational Software
No effect!The study described was a randomized controlled trial, and showed that the software did not make a noticeable difference in any of the categories. It did not help or hurt with 1) early reading (first grade), 2) reading comprehension (fourth grade), 3) pre‐ algebra (sixth grade), or 4) algebra (ninth grade) (19).
How did you do?
If you got 7-8 right, there’s less than a 1% chance you were guessing. If you got 5-6 right, there was only an 8.5% chance you were guessing, so it might be skill. If you got 1-4 right, then you did no better than randomly guessing. If you got zero right … we could get useful information by always doing the opposite of what you do.
The effects of social interventions are extremely complex. All of these programs sound good, but unintended consequences can get in the way. It’s very difficult to work out what’s going to be successful ahead of time. Instead, we need to test, measure the results, and take it from there.
I thought Round 2 would have no effect and expected Round #5 to have no effect not a negative one, I got 6 out of 8 correct. How well did you do?
I recommend checking out the links and references. Gwern’s comment there was also interesting.
So out of this sample, the only two interventions that had positive effects were based on one-on-one relationships. Any wisdom we can draw from this, or is it just a coincidence?
Yeah, I noticed that as well.
In fact, I had updated in that direction by about halfway through the exercise (based on weak one-of-these-things-is-not-like-the-others evidence), and on that exact basis, decided that of 5, 6, 7, and 8, only #7 was likely to help.
Probably one-on-one relationships are just higher-quality inputs. One person interacting with one other person can pay a lot of attention to subtle cues and signs of misunderstanding (and thus adjust the interaction appropriately) in ways that we haven’t figured out how to automate yet.
In other words, it works for the same reason that we don’t have something that can reliably pass the Turing test over a long period of time yet: manipulating people is complicated, and humans are optimized to do that exact task.
I remember reading somewhere (does anyone remember where?) that while there’s little evidence for one school of psychological or psychiatric therapy over another, there is strong evidence that spending a lot of time with an intelligent, sympathetic listener is good for you.
I suppose this means I should look into private lessons/tutors for skills I want to learn, whenever I can afford them.
Me too. I’ve also read that teenage girls are as effective as professionally trained therapists… and even that you might not need a human listener at all to get the benefits; just keeping a diary helps.
http://lesswrong.com/lw/2j/schools_proliferating_without_evidence/ and http://lesswrong.com/lw/94t/meta_analysis_of_writing_therapy/ seem relevant.
If you want more detailed refs including the claim that the particular school matters much less than the quality of the therapist (with an honorable exception for CBT), http://counsellingbooks.com/bibliography/theory-and-research/general-and-comparative.html#LambertBergin_1994 http://counsellingbooks.com/bibliography/theory-and-research/general-and-comparative.html#BeutlerEtAl_1994 are relevant.
All kinds of therapists or just traditional psychoanalysts? The efficacy of cognitive behavioral therapy is supposed to have strong empirical support. I wouldn’t be very surprised if it turned out that most of that support is critically flawed but I also wouldn’t be surprised if the study you’re referring to was behind the times.
Dunno, it’s just something that I remember hearing once. I couldn’t find a source.
I did manage to come across the name for this effect: Dodo Bird Verdict.
Probably just traditional psychoanalysis. I mean, lying on the couch and discussing sexual topics, is there something a teenager wouldn’t be able to do? On the other hand, rational thinking and proper behavior, that’s like an exact opposite of being a teenager. :)
I got all 8 correct. My take is that programs that make participants feel like they have low status are unlikely to succeed.
Are you sure? The Scared Straight result definitely matches that, but on the others the connection between status and efficacy is less clear to me. Nurse-Family, DARE, Even Start, and Big Brothers/Sisters seem low-statusy, while 21st Century Learning Centers and Educational Software don’t, but the results don’t line up that way.
That seems wrong, after all, if you look at history, it’s clear that making people feel like they have low status is a great way to control them.
Which might be relevant if the programs were rated on how well they controlled a populace, rather than in their impact on life outcomes.
I would explicitly say “The answer is yes for some but not all of these”—ie it’s not a trick question where the answer is that they all worked, or none did.
I’ve edited the OP to include this.
I think there’s a missing word there.
Cool—thanks! First time I did this I kind of skimmed through, and only when I read the second one and saw that it was different from the first did I go back and try to make guesses, so it’s helpful to know in advance.
Seconded; I didn’t even try guessing because I didn’t think that getting a score of correct answers was going to be the point.
I did “just for fun”. (I got two of them right, in six of them I guessed no effect and there was some effect or vice versa, and in none of them I guessed a negative effect and there was a positive effect or vice versa.)
As it usually happens in the social “sciences,” it’s very naive to believe that in any of these cases we have anything like solid evidence about the total effect of the programs in question. Even ignoring the intractable problems with disentangling all the countless non-obvious confounding variables, there is still the problem of unintended consequences—which may be unaccounted for even if the study seemingly asks all the relevant questions, and which may manifest themselves only in the longer run.
Take for example this nurse-family partnership program. Even if the study has correctly proven that these positive outcomes have occurred in the families covered by the intervention, and that they are in fact a consequence of the intervention—a big if—we still have no way of knowing its total long-run effect. For one, it may happen that it lowers the cost of having children for poor unmarried women, both by providing assistance and by lowering the stigma and fear of such an outcome, so that in the new long-term equilibrium, more children are born to such women, especially the least responsible, resourceful, and competent ones, eventually increasing the total measure of child poverty, neglect, abuse, etc. Of course, this may or may not be the case, but there’s no way to know it based on these studies that purport to give a definitive evaluation of the program’s success.
(Don’t be mistaken, I see single motherhood as a serious problem, although not an insurmountable one—it was high in the post-war USSR, yet that generation turned out well enough, considering all the deplorable circumstances.
What alarms me is how the proposed practical response to it is always either status punishment of, by and large, downtrodden and psychologically unwell women, or even more aggressive measures. Perhaps liberals are to blame for this. No, seriously, I mean the fact that in the West liberal movements have been neglecting the issue except for ivory-tower talk, and gave the hard Right a monopoly on proposing solutions.)
Well, it’s not like we have no evidence either way. We have weak evidence for a positive effect.
It may also happen that people in dangerous and impoverished situations pursue early and fecund reproductive strategies: if you can’t count on each child surviving and prospering, then you have more kids (and start earlier) to increase the chance of some child surviving and prospering. In this case, lowering the risks to children and mothers would result in fewer children.
I find it exceedingly unlikely that increasing “stigma and fear” will reduce such behavior. For instance, out-of-wedlock births, teen pregnancy, divorce, etc. are all higher in more socially conservative societies — including when we compare the U.S. vs. Western Europe, or “red states” vs. “blue states” within the U.S. …
I found this article interesting overview of examples of unintended consequences of past changes, that makes a case for being very cynical of this particular kind of argument:
A Really, Really, Really Long Post About Gay Marriage That Does Not, In The End, Support One Side Or The Other
The post you linked has almost nothing to do, really, with gay marriage, but it’s bang-on about how people respond to incentives. The warning against the typical mind fallacy (in the article, phrased as thinking of how you would respond to an incentive rather than the marginal case) is also highly relevant.
Oh yes I fully agree. I wasn’t trying to start a discussion on gay marriage after all! Its just the title the author chose.
I think it’s interesting that gay-friendly states in the U.S. have lower heterosexual divorce rates than gay-hostile states. Reconstructing causation here tends to be pretty tricky; there are lots of confounding factors and lots of people want to put the blame for their own circumstances on their rivals.
It’s amusing to see a libertarian suggesting that it’s probably good for the few to suffer for the sake of the many. And interesting to see that the interests of existing illegitimate children are not noticed—it’s assumed that their mothers are the only people worth mentioning.
Eh. There are two camps in libertarianism: the moral libertarians, and the technical libertarians. The moral libertarians derive their policies from principles- force is wrong, taxation implies the threat of force, and thus we need to build a society without taxation if we want to live in a moral society.
The technical libertarians derive their policies from economic arguments and history. It doesn’t matter whether you think it’s moral or immoral to lend money for profit- let’s look at societies which allow that and societies which don’t, and see which ones prosper more, and apply theoretical principles to expect which should be the case.
And so the atheist moral libertarian looks at gay marriage, and says something along the lines of “the state shouldn’t be involved in marriage at all!” or, if you’re lucky, “the state should recognize a marriage contract between any two consenting adults!”. (The Christian moral libertarian probably thinks that gay marriage is wrong for the standard Christian reasons.) The technical libertarian, though, will be willing to ask about the tradeoffs involved- sure, allowing gay marriage makes gays happier (and, if the institution of marriage alters their sexual practices, probably healthier too), but what does it do to the rest of society? When the little boy breaks a glass window, the glazier is happier too, but we need to consider both the seen and the unseen.
Which is what I thought the article was mostly about- libertarians are very quick to jump to complicated and far-seeing analyses for economic issues, and are sometimes reluctant to do them for social issues. But if you’re a libertarian because you think it’s what history and theory tell us will be best, then when considering what will be best on another issue, perhaps you should apply some history and theory!
On a related note: For Many Poor Black Girls, Teen Pregnancy is a Rational Choice
It seems like you’re losing focus of my point. I am merely trying to demonstrate that it’s wrong to consider studies of this sort as solid and conclusive evidence about the overall effects of the social interventions under consideration. I mentioned this scenario only as one plausible way in which one of these studies could be grossly inadequate, not as something I’m trying to prove to be the case.
I suppose I don’t find it to be particularly plausible. Moreover, it seemed that you were discounting the study as offering any evidence at all regarding long-term effects; whereas it seems to me that short-term effects offer weak evidence regarding long-term effects. If we know something is short-term beneficial, that isn’t strong evidence of it being long-term beneficial — but it isn’t evidence of it being long-term harmful either.
It’s worth it to keep looking — I certainly agree that it’s a failure of many social interventions to look at only short-term effects, especially when this failure is iterated. That’s where we get Campbell’s Law from.
(That said, it would be really surprising if deeper investigation of social reality happened to closely confirm the preconceived notions of one particular political faction. I mean, seriously, why that one?)
I find it very likely that they will since social shaming is among the most powerful means a culture can employ to maintain norms.
Blue state vs. Red state comparisons as well as Western Europe vs. USA are weaker than they seem because of demographics differences. The US Black population was particularly hard hit by the fallout of the sexual revolution, partially leading to the infamous circumstances in the US inner cities. Also note that the heavily shame based groups such as say the Amish or the Mormons in the US maintain very low rates of such dysfunction.
We clearly also clearly see that all Western societies used to have far fewer unwed mothers, less divorce and teen pregnancy when these where more strongly shamed before the sexual revolution. Obviously empirically observed covariation is a necessary but not sufficient condition for causality.
But in the light of what else we know of humans I’m pretty sure there is causality there.
This doesn’t seem to be a relevant response.
For instance, take divorce. This paper notes that “[t]he red states have residents with lower mean levels of education, younger ages at marriage, quicker transitions to the first birth, higher hazards for subsequent births, lower rates of maternal labor force participation, and lower family incomes” — all traits which correlate with divorce risk. But even after controlling for race, age, income, age at first marriage, and Southern ethnicity (!), areas with a higher proportion of conservative Protestants still have higher divorce rates. “The average county would almost double its divorce rate as its proportion [of conservative Protestants] moved from 0 to 100 percent.”
At least in the case of divorce, it sure looks like sex-shaming culture produces the dysfunctions that it shames. As you note, we can’t be sure — and it’s easy to mistake hypotheses that are raised to attention by our preconceptions (confirmation bias) for hypotheses that are actually compelled by the data. A racist conservative is inclined to see racist conservative patterns; a progressive libertarian is inclined to see progressive libertarian patterns. We have to actually care about reality to find out what reality says.
It seems that would be kind of difficult to measure. I am reminded of the claims by Mahmoud Ahmadinejad that Iran does not have any gay people.
Difficult not only to measure, but even to bring the meanings behind these measurements to consistency. Were there many couples who’d get divorced nowdays, but lived in a formal marriage while not being on speaking terms? Were there many fathers and stepfathers (forced into marriage after unintended pregnancy, etc) whose parenting had a worse effect than unwed motherhood would?
I suspect all that, and more, but I have no way to prove it. A society’s facade—especially that of a shame-based, traditionalist culture—can be practically impenetrable once the witnesses fade away. We only have a strong image of Victorian philistinism and hypocrisy because a Victorian (and, mostly, Edwardian) elite attacked it vigorously. Today, most people have a cached belief that the Edo-era samurai were an uniformly honorable aristocracy obsessed with Bushido—but I’ve heard from several sources that it was mostly propaganda (both contemporary and Imperial one), and that most samurai behaved like glorified thugs whenever they could get away with it.
[Dear downvoters, how about a rebuttal? Use a sockpuppet if you want, just tell me whether you do, in fact, have a reason to punish this comment on its own weight. I do want more skill at epistemic rationality, and would really benefit from being showed a flaw where I thought there were none.]
For that matter, I’ve read claims that if you read diaries by Victorian women, you find that a lot of them liked sex and didn’t feel bad about liking it.
Why was this downvoted?
I’ve been noticing a lot of my comments get rapidly downvoted once shortly after I post them lately, especially (but not exclusively) in threads where I post libertarian-progressive-ish rebuttals to social-conservative positions.
I’d like to think that it’s just someone who doesn’t approve of political discussion on LW — but the socially conservative interlocutors don’t seem to be getting the same treatment. (With the exception of the ever-popular sam0345, whose low comment scores I expect have more to do with his hostile attitude than the fact that he posts about politics.)
So there does seem to be some Blue/Green unpleasantness going on here. Comments advocating “race realism”, sexual shame, or other socially conservative positions tend to float around +3 or +4, while responses disagreeing with them — even with citations to academic work and evidence on the subject — tend to float around −1 to +1.
It doesn’t bother me all that much. If my comments were actually getting buried, I’d be worried that we had a bury brigade going on — but they’re not. My current hypotheses are either ⓪ I’m just not very good at commenting, ① I have a stalker, ② the idea that social conservatism is “contrarian” really gets some folks excited, or ③ social conservatives think it’s worthwhile to downvote comments that disagree with them. If it’s the latter, well, I suppose all I can do is mention that I don’t downvote interesting comments that I reply to, and ask them to extend the same courtesy.
(I don’t mind if this comment gets downvoted, by the way. I feel uncomfortable with discussions of the voting system, as they can become a meta rathole.)
EDIT: OH GHODS, PEOPLE, STOP UPVOTING THIS. YOU’RE CREEPING ME THE FUCK OUT.
I think this is what being on one side of a tribal conflict looks like from the inside. My experiences have been similar, with many of my posts getting instantly down voted to −3 to −4, then slowly recovering karma later. As you probably recall from our recent conversations with me we have differing opinions on some politically charged subjects.
I don’t think you a bad poster and you seem to have a high karma score so we can mostly throw out ⓪. I recall often up voting posts by you, even the ones I disagree with and only recall downvoting a recent one where you seemed to be plain wrong in the context of the discussed article. In that case I also made a comment explaining why I thought it wrong. The contrarian explanation as I will elaborate later may have some truth to it. Explanation ③ seem far fetched considering social conservatives are such a tiny minority of the readership and can be discounted as an explanation for what you say you experience. Of these explanations I think ① is the most likely. I think any of us talking about politics regardless of our positions probably eventually catch the attention of someone who feels like throwing a hissy fit. Right leaning posters have complained of people going through their comment history and down voting every post they’ve ever made. I’ve experienced such karmassasiantions in the past too.
Now having said this there have been signs of escalating tensions. Posters have been saying they feel more and more unwelcome and I can totally see why since there are more and more posts that signal “liberal” tribal affiliations. Some like the article criticized by the links I gave are pretty blatant about this. Even some old time well respected posters like Yvain have recently been called out on not being bothered to avoid dog-whistling affiliations.
Now obviously you have some right wing digs like that in recent articles and they may be escalating too, but they are of a more alt-right not conservative nature. And yes any kind of alternative right sentiment, be it Moldbugian Neoreaction or consistent Paleoconservatism is basically being an intellectual hipster. This brings us back to ③ and I think also explains why left leaning users like Multiheaded fear they are losing the battle of ideas.
If due to such superior intellectual fire power LessWrong ever got even 10% of conservative readership (still a tiny minority), the metacontrarians would probably cycle back to an exotic form of liberalism. And if that exotic form reached 10%, I’m betting some kind of libertarianism would be back in vogue… I need to again emphasise for the reader who didn’t follow the link that where something lands on the metacontrarian ladder does not tell us its truth value.
Now this kind of cycling is I think mostly self-corrective, since it is an intellectual fashion. The real problem in my mind is how political identification can create and escalate conflict between these somewhat shifting fads.
This. Posters should be encouraged to avoid down voting just political comments they disagree with. Also I think putting more emphasis on keeping your identify small or even apolitical might do us good.
Failing all this I think we really should consider if the overly-strictly interpreted no mindkillers rule that was prevalent as little as a few months ago that much reduced political discourse was a good thing that should be restored.
Don’t be freaked out. People politely complaining about being down voted seem to always get up voted on LessWrong. :)
I used to be excited about the idea of harnessing the high intellectual ability and strong norms of politeness on LW to reach accurate insight about various issues that are otherwise hard to discuss rationally. However, more recently I’ve become deeply pessimistic about the possibility of having a discussion forum that wouldn’t be either severely biased and mind-killed or strictly confined to technical topics in math and hard sciences.
It looks like even if a forum approaches this happy state of affairs, the way old Overcoming Bias and early LessWrong arguably did for some time, this can happen only as a brief and transient phenomenon. (In fact, it isn’t hard to identify the forces that inevitably make this situation unstable.) So, while OB ceased to be much of a discussion forum long ago, LW is currently in the final stages of turning into a forum that still has unusual smarts and politeness, but where on any mention of controversial issues, battle lines are immediately drawn and genuine discussion ceases, just like elsewhere. (Even if the outcome may still look very calm and polite by the usual internet standards.)
The trouble is, the only way a “no-mindkillers” rule can improve things is if it’s done in an extreme form and with ruthless severity, by reducing the permissible range of topics to strictly technical questions in some areas of math and hard science and consistently banning everything else. The worst possible outcome is to institute a partial “no-mindkillers” rule, which would work under a pretense that rational and unbiased discussion of a broad range of topics outside of math and hard sciences is possible without bringing up any controversial and charged issues, and without giving serious consideration to disreputable and low-status views. This would lead to an entrenched standard of cargo-cult “rationality” that incorporates all the biases, delusions, and taboos of the respectable opinion wholesale, under a pretense of a neutral, pragmatic, and unbiased restriction of irrelevant and distracting controversial topics.
Thus, it seems to me like the only realistic possibilities at this point are: (1) increasing ideological confrontations and mind-killing, (2) enforcement of the above-described cargo-cult rationality standards, and (3) reduction of discussion topics to strictly technical questions, backed by far stricter, MathOverflow-type standards. Neither of these looks like a fulfilment of LW’s mission statement, but (2) seems to me like the worst failure scenario from its point of view.
Uh huh. I fully endorse your analysis. Except that I’d say (1) would still leave us far better off than the typical confrontation-allowed political forum out there, because LWers would probably at least be willing to state their positions clearly, and would accept help in clarifying/refining those positions—even if the art of changing one’s mind should be lost, LW discussion would still retain some value. So I’d rather have (1) than (3).
Please consider that both Torture vs Specks and Three Worlds Collide are, as it seems to me, very much political—indeed, the latter could be construed as today’s very Blue vs Green with a touch of imagination, yet the debates on those have been quite OK.
I don’t think so. At least I can’t figure out which side is supposed to be which.
Me neither. Maybe we lack the imagination...
Are you concluding too hastily about the cause of deterioration? In the early days, OB had two major voices with conflicting ideologies. I think that’s what lent it greater intellectual excitement. I do not think it a matter of ideological alignments being absent in the golden age. Rather, it allowed space for discussion of fundamental differences—as opposed to the analysis down to quarks of the highly obvious that’s taken front seat today.
Consider this old posting on OB. The level of objectivity wasn’t higher, but the level of engagement with fundamental issues was.
If I remember correctly, you replied to several of my comments on fairly controversial topics recently, but for the record, I didn’t downvote any of them. I downvote direct replies to my comments only if I believe that someone is arguing in bad faith, or when I’m annoyed with some exceptionally bad failure of basic logic or good manners.
Cool, thanks. Good to know.
FWIW, I can recall upvoting at least one of yours which had previously been at −1.
Aargh, meta rathole …
BTW… A few days ago I browsed through all comments I had posted in the past few months, and it looks like most of the comments that weren’t well received back then have since been upvoted to +1 to +3. How comes?
Damned if I know.
What’s more frustrating is getting upvotes, but they’re not for recent comments. I’d really like to know what’s attracting attention. And get notifications for comments on my posts.
Being new here, I’ve upvoted some comments on the original Sequences posts that were particularly insightful. Some of them were much more recent than the posts but probably still a year or two old. I did this under the assumption that other new people may be liable to read only the highly-rated comments on those original posts.
This is more likely to be a good model of how people’s reading habits work on newer posts than on old Sequence posts. Posts imported from Overcoming Bias have the comments in chronological order.
Yes. I don’t even pay attention to the last two digits in my total karma score anymore.
Aw, c’mon. I quip about it semi-seriously, and then you post all these dark suspicions in plain language! It’s no fun this way; don’t you want to feel persecuted by an evil conspiracy every once in a while? I demand that the ideological downvoting continue!
Vast right-wing conspiracy, dude. (EDIT: meant largely as self-deprecating humour!)
Maybe fear and stigmatization do work, but the less socially conservative societies use other means of maintaining cohesion, ones that are more palatable to them?
The most, as they say, disfunctional groups in a society—such as ghetto youth—have rather illiberal and inegalitarian norms anyway, it’s just that those norms involve violent and parasitic behavior rather than that of a decent, productive conservative community.
I’d take liberal arguments about the ineffectiveness of “stigma and fear” much more seriously if those liberals weren’t simultaneously using “stigma and fear” to promote their own agenda, e.g., suppressing discussions of race realism.
I think I’m having difficulty understanding your comment. It sounds to me as if you are morally equating threatening the lives and health of impoverished women and children, to disagreeing with (or even downvoting) someone in an Internet forum. That confuses me, so I conclude that one of my beliefs regarding your comment is fiction. Please clarify.
Also, I find it awkward that you seem to be characterizing my comments as “liberal”, since that word seems to be commonly used to mean anything from the center-right (e.g. the Democratic Party), to the Greens, to Euro-style social democrats. I think of myself as a center-libertarian, which is where the Political Compass places me as well. Unlike some anti-authoritarians, I take anti-authoritarianism as logically entailing feminism and a critical approach to gender, race, and other topics beloved by many progressives; I’d rather cheerfully identify as a progressive libertarian if I thought anyone had a chance of understanding what I meant by that.
Who said anything about moral equating? I’m trying to determine the effect of social pressure, what you called “stigma and fear” on behavior.
(Edit: Another way to phrase this is that you may be confusing the statements ‘I find it exceedingly unlikely that increasing “stigma and fear” will reduce such behavior.’ and ‘We shouldn’t attempt to use “stigma and fear” to reduce such behavior.’)
Try advocating race realism or some other politically incorrect position outside an anonymous internet forum and you’ll quickly discover the consequences are much more serious and you’re likely to loose your job at the very least.
My main point is that while left-wingers claim to believe that stigmatizing undesirable behaviors is ineffective, they don’t act like they believe it.
That’s a good point.
I’m curious what sort of procedures might use “stigma and fear” to reduce unwed teen pregnancy in poor women. If we take seriously the article CronoDAS posted regarding the rational motivations for young poor women having babies, then presumably addressing those specific motivations might do it.
(Notably, we would not expect preaching traditionalist views via religion, or other means that did not change the material utility landscape, to work. If, as the article holds, young women choose to have babies on the basis of their material expected outcomes, then the procedures would have to alter the young women’s material expected outcomes; and — to qualify as relevant here — would have to do so using stigma and fear.)
After thinking about it for a bit and coming up with some possible procedures for doing so, I’ve decided not to post most of them because they’re really quite unpleasant; they’re the sort of things that would occur in dystopian fiction. I’ll just give one example: changing the landscape for infant mortality by making it illegal for physicians to attend births to unwed mothers.
On second thought, I don’t think I am confusing those statements you mention — I think they’re both true. First, using stigma and fear to change the motivations for teen mothers would not work, primarily for political reasons (e.g. physicians would not put up with it; people would revolt; etc.) And second, it would be immoral to try; especially given that the same motivations could be addressed in non-dystopian ways.
The same could be said for a lot of other views; that’s scarcely unique to one end of a political spectrum. For instance, there is a long history of people losing their jobs for advocating labor unionization, even in the presence of laws forbidding employers from firing workers for doing so.
Outside of explicitly political views: Advocating pederasty would probably not get you a whole heck of a lot of friends or willing coworkers either; nor would advocating for the right of parents to kill disobedient children, as found in various ancient civilizations. In both cases people might reasonably conclude that they (or their children, etc.) were unsafe around a person who held such views. Might that be the case with racialism, too?
While the motivations described to the article are in some sense rational, they’re rational in an adaptation executor kind of way. Thus, other adaptations, e.g., the desire to avoid social shaming, can be used to contract them.
Compare the situation today with the situation 60 years ago. Notice what is and isn’t different.
So you agree that “stigma and fear” is in fact an effective way to change people’s behavior.
It’s an interesting thought. How would you control for the baseline levels of social shame that racial minorities, poor people, and teen mothers are already subjected to?
All sorts of things. There’s a lot less lead paint on the walls, for instance; a lot more black men in prison for victimless crimes, too; “the situation” is hardly simple. What leads you to single out your particular claim?
Running a power drill through the eyeballs of people you dislike might “change people’s behavior” too, but that doesn’t make it a policy proposal rather than a psychopathy symptom.
In other words, it isn’t clear that this is the same sort of thing at all. And specifically, the reasons certain proposals wouldn’t work have to do with political realities — e.g. if you try to generate “stigma and fear” by forbidding doctors from treating teen mothers, the doctors are not likely to cooperate.
Specifically, I was referring to the fact that Blacks were even poorer back than so following the argument CronoDAS’s link was making we would expect there to be more unwed motherhood among them. On the other hand unwed motherhood was much more stigmatized (and no it didn’t involve laws forbidding doctors from treating teen mothers).
This is unrelated to the main argument, but I’m not sure the crimes in question are truly victimless for reasons given in these two posts.
This is getting too political, best stop now or re-frame in a non-divisive way.
This sounds like the fallacy of gray. You can seem sophisticated by saying that it is naive to take any research at face value—that there are always limitations to the studies, and possible unmeasured effects. But there are some cases where the evidence is stronger than others, and where the existence of large unmeasured effects in the opposite direction is less plausible. Paying attention to that evidence is updating.
The particular case of Nurse-Family Partnership looks like one of more strongly supported programs. It has been studied extensively using randomized controlled trials, with data collected on a variety of outcomes (including a follow-up study looking at 19-year-olds who had been through the program as infants). GiveWell named it a top-rated US organization in 2010 (they now say that it is outstanding but lacks room for more funding). Your story about a possible hidden cost does not sound very plausible, and I imagine that some of the studies include data which speak to some of the steps in your story (e.g., whether women who take part end up having more additional children than those who do not).
But somehow I can guess that you do trust the negative results shown… right?
I’m not sure what exactly you’re trying to imply with this comment. You have complained that I was reading your comments too uncharitably in the past, so I’m trying to interpret it as something other than a taunt, but without success.
I found this an interesting overview of examples of unintended consequences.
A Really, Really, Really Long Post About Gay Marriage That Does Not, In The End, Support One Side Or The Other
Much of the uncertainty in estimating the “success” of these programs lies in not knowing to what degree each of the different social indicators measuring said “success” have already been corrupted, in line with Campbell’s law.
Seeing the phrase “randomized controlled trials” over and over was at least a nice start. Too much social policy debate seems to compare Group A with Policy A and Group B with Policy not-A as if “A vs not-A” wasn’t merely one out of thousands of major economic and cultural difference between the two self-selected groups.
Agreed on RCTs being the gold standard, the top tier of the evidence hierarchy (other than reviews of multiple RCTs).
However, if you choose a biased (corrupted) indicator, all your perfect measurement methodology can only minimize additional biases. A top of the line flood-proof house that you erect on top of a swamp will still go down. (Substitute with weakest-link analogy of your choice.)
I would like to see this article published in the (very hypothetical) World Where Nobody Just Guesses. They would surely be amused at that sky-high 1% figure, when the real number is obviously 0% since nobody was going to guess.
You can’t just flip around statements about conditional probability, darn it.
Did the first 4, then skipped to the answers—and I got all of those right! Well, with the exception of DARE; I see no logical way it can’t be harmful and counterproductive—for the simple reason that it’s crying wolf to teenagers, who are both impressionable and suspicious of adult authority.
I think that, when programs like these are unleashed upon sheltered teenagers from middle-class backgrounds (which is now happening in Russia as well), it can be a big part of the problem; kids are told in strong terms that marijuana and meth are both deadly and awful. Then they personally see drug users who are functioning people (which are probably the majority), and apply reversed stupidity to conclude that adults just hate fun, and that hard drugs must be an excellent idea. My BF got an article published on this very issue, actually.
P.S.: check out another article, published in a very unexpected venue. Apparently, people who have some good sense and aren’t in underclass conditions might not suffer badly from heavy lifelong drug use.
What does it say about me that on reading #1 I thought “Well, I guess that it increased the rate of crime among males and decreased it among females, but given that more crimes are committed by males than by females to begin with, I guess the total crime rate went up”?
Here are my results. P(yes) is the probability I gave that a given intervention worked before I saw the answers.
My average probability rating was 0.39. My average probability rating for correct answers was 0.5, and my average probability rating for incorrect answers was 0.35.
If I do better than chance at this, it’s not by much.
6⁄8 - I thought 21st Century schools would improve behavior, and I thought Even Start would improve child literacy but not adult literacy. This is really more of a 5⁄7, though, since I already knew of DARE’s ineffectiveness.
The Nurse-family partnership thing sounds like a big-government liberal dream program, so how could I not predict that it would greatly improve outcomes :P
The Nurse-family partnership sounds to me more like personalized care, which is a big deal in terms of effectiveness. It doesn’t say it was run by the government, and it read to me like it was done by individual hospitals.
Oh, so it is. It’s a major-foundation-funded charity, my bad.
According to the Wikipedia article on happiness economics, historically women’s have reported greater happiness than men up till the 1960s. This coincides with the triumph of first wave feminism—suffrage an legal obstacles and the emergence of social justice feminism that’s more retributory and provides retaliatory privellages for women. Since then, women have reported increasingly less happiness and are no more unhappy than men on aggregate, in the West. Evidence supporting social conservatism?
2⁄8 here...
Did they miss “against a properly set up control” after the word “test”?
Dear OP,
I applaud your effort to promote clear thinking on effects of policy. One thing I wanted to mention is that even programs with weak effects may be eventually worthwhile if there is some cancellation of direct effects of the program by some indirect effects. Even if not worthwhile directly, interventions with “effect cancellation” may perhaps be informative for designing future interventions. People in “mediation analysis” worry about these things a lot.
Analyzing mediation is tricky because it needs strong assumptions.