I, like many others, was very amused at the structure of the MONETARY AWARD.
I’m not sure it was an advisable move, though. There’s an ongoing argument about the effect of rewards on intrinsic motivation. But few argue that incentives don’t tend to incentivise the behaviour they reward, rather than the behaviour the rewarder would like to incentivise. In this instance, the structure of the reward appears to incentivise multiple submissions, which I’m pretty sure is not something we want to happen more.
In some contexts you could rely on most of the participants not understanding how to ‘game’ a reward system. Here, not so much, particularly since we’d expect the participants to know more game theory than a random sample of the population, and the survey even cues such participants to think about game theory just before they submit their response. Similarly, the expectation value of gaming the system is so low that one might hope people wouldn’t bother—but again, this audience is likely to have a very high proportion of people who like playing games to win in ways that exercise their intelligence, regardless of monetary reward.
So I predict there will be substantially more multiple submissions this time compared to years with no monetary reward.
I’m not sure how to robustly detect this, though: all the simple techniques I know of are thwarted by using a Google Form. If the prediction is true, we’d expect more submissions this year than last year—but that’s overdetermined since the survey will be open for longer and we also expect the community to have grown. The number of responses being down would be evidence against the prediction. A lot of duplicate or near-duplicate responses aren’t necessarily diagnostic, though a significant increase compared to previous years would be pretty good evidence. The presence of many near-blank entries with very little but the passphrase filled in would also be very good evidence in favour of the prediction.
(I used thinking about this as a way of distracting myself from thinking what the optimal questionnaire-stuffing C/D strategy would be, because I know that if I worked that out I would find it hard to resist implementing it. Now I think about it, this technique—think gamekeeper before you turn poacher—has saved me from all sorts of trouble over my lifespan.)
I took the survey.
I, like many others, was very amused at the structure of the MONETARY AWARD.
I’m not sure it was an advisable move, though. There’s an ongoing argument about the effect of rewards on intrinsic motivation. But few argue that incentives don’t tend to incentivise the behaviour they reward, rather than the behaviour the rewarder would like to incentivise. In this instance, the structure of the reward appears to incentivise multiple submissions, which I’m pretty sure is not something we want to happen more.
In some contexts you could rely on most of the participants not understanding how to ‘game’ a reward system. Here, not so much, particularly since we’d expect the participants to know more game theory than a random sample of the population, and the survey even cues such participants to think about game theory just before they submit their response. Similarly, the expectation value of gaming the system is so low that one might hope people wouldn’t bother—but again, this audience is likely to have a very high proportion of people who like playing games to win in ways that exercise their intelligence, regardless of monetary reward.
So I predict there will be substantially more multiple submissions this time compared to years with no monetary reward.
I’m not sure how to robustly detect this, though: all the simple techniques I know of are thwarted by using a Google Form. If the prediction is true, we’d expect more submissions this year than last year—but that’s overdetermined since the survey will be open for longer and we also expect the community to have grown. The number of responses being down would be evidence against the prediction. A lot of duplicate or near-duplicate responses aren’t necessarily diagnostic, though a significant increase compared to previous years would be pretty good evidence. The presence of many near-blank entries with very little but the passphrase filled in would also be very good evidence in favour of the prediction.
(I used thinking about this as a way of distracting myself from thinking what the optimal questionnaire-stuffing C/D strategy would be, because I know that if I worked that out I would find it hard to resist implementing it. Now I think about it, this technique—think gamekeeper before you turn poacher—has saved me from all sorts of trouble over my lifespan.)