I think you may be misunderstanding why people focus on selection mechanisms. Selection mechanisms can have big effects on both the private status returns to quality in comments (~5x) and the social returns to quality (~1000x). Similar effects are much less plausible with treatment effects.
Claim: selection mechanisms are much more powerful than treatment effects.
I think people are using the heuristic: If you want big changes in behavior, focus on incentives.
Selection mechanisms can make relatively big changes in the private status returns to making high quality comments by making high quality comments much more recognized and visible. That makes the authors higher status, which gives them good reason to invest more in making the comments. If you get 1000x the audience when you make high quality comments, you’re going to feel substantially higher status.
Selection mechanisms can make the social returns to quality much larger by focusing people’s attention on high quality comments (whereas before, many people might have had difficulty identifying high quality even after reading it).
“More powerful” seems like it’s implicitly using categories that don’t cut at the joints. I think Aceso Under Glass’s post on Tostan makes an important distinction between capacity-building and capacity-using interventions:
This is more speculative, but I feel like the most legible interventions are using something up. Charity Science: Health is producing very promising results with SMS vaccine reminders in India, but that’s because the system already had some built in capacity to use that intervention (a ~working telephone infrastructure, a populace with phones, government health infrastructure, medical research that identified a vaccine, vaccine manufacture infrastructure… are you noticing a theme here?). [...] Having that capacity and not using it was killing people. But I don’t think that CS’s intervention style will create much new capacity. For that you need inefficient, messy, special snowflake organizations.
I’d guess that treatment effects seem less powerful than selection effects of equal importance because treatment effects are typically more capacity-building loaded.
I think you may be misunderstanding why people focus on selection mechanisms. Selection mechanisms can have big effects on both the private status returns to quality in comments (~5x) and the social returns to quality (~1000x). Similar effects are much less plausible with treatment effects.
Claim: selection mechanisms are much more powerful than treatment effects.
I think people are using the heuristic: If you want big changes in behavior, focus on incentives.
Selection mechanisms can make relatively big changes in the private status returns to making high quality comments by making high quality comments much more recognized and visible. That makes the authors higher status, which gives them good reason to invest more in making the comments. If you get 1000x the audience when you make high quality comments, you’re going to feel substantially higher status.
Selection mechanisms can make the social returns to quality much larger by focusing people’s attention on high quality comments (whereas before, many people might have had difficulty identifying high quality even after reading it).
“More powerful” seems like it’s implicitly using categories that don’t cut at the joints. I think Aceso Under Glass’s post on Tostan makes an important distinction between capacity-building and capacity-using interventions:
I’d guess that treatment effects seem less powerful than selection effects of equal importance because treatment effects are typically more capacity-building loaded.