Better: randomly select a group of users (within some minimal activity criteria) and offer the test directly to that group. Publicly state the names of those selected (make it a short list, so that people actually read it, maybe 10-20) and then after a certain amount of time give another public list of those who did or didn’t take it, along with the results (although don’t associate results with names). That will get you better participation, and the fact that you have taken a group of known size makes it much easier to give outer bounds on the size of the selection effect caused by people not participating.
You can also improve participation by giving those users an easily accessible icon on Less Wrong itself which takes them directly to the test, and maybe a popup reminder once a day or so when they log on to the site if they’ve been selected but haven’t done it yet. Requires moderate coding.
I would find such a feature to be extraordinarily obnoxious, to the point that I’d be inclined to refused such a test purely out of anger (and my scores are not at all embarrassing). I can’t think of any other examples of a website threatening to publicly shame you for non-compliance.
Yep! But it’s the best way I can imagine that someone could plausibly create on the forum.
Better: randomly select a group of users (within some minimal activity criteria) and offer the test directly to that group. Publicly state the names of those selected (make it a short list, so that people actually read it, maybe 10-20) and then after a certain amount of time give another public list of those who did or didn’t take it, along with the results (although don’t associate results with names). That will get you better participation, and the fact that you have taken a group of known size makes it much easier to give outer bounds on the size of the selection effect caused by people not participating.
You can also improve participation by giving those users an easily accessible icon on Less Wrong itself which takes them directly to the test, and maybe a popup reminder once a day or so when they log on to the site if they’ve been selected but haven’t done it yet. Requires moderate coding.
I would find such a feature to be extraordinarily obnoxious, to the point that I’d be inclined to refused such a test purely out of anger (and my scores are not at all embarrassing). I can’t think of any other examples of a website threatening to publicly shame you for non-compliance.