There are certainly good ways to ask such a question with reasonable motivation to get it right. You could include this in a 5-question quiz, for instance, and say “you get paid only (or more) if you get 3 correct”. And then vary and permute the questions so nobody uses 20 accounts to do the same task and not separately answer the questions.
But that’s expensive and time-consuming, and unless the paper specifies it, one should assume they did the simpler/cheaper option of just paying people to answer.
Monetary incentives raise solution rates a little, but not that much. Lex Borghans and co-authors manipulate small incentives and they do almost nothing. Ben Enke and co-authors offers a full month’s salary to Nairobi based undergrads and finds a 13% percentage point increase in solution rates.
I’m not sure how our manipulations would interact with monetary incentives. But I’d like to know!
There are certainly good ways to ask such a question with reasonable motivation to get it right. You could include this in a 5-question quiz, for instance, and say “you get paid only (or more) if you get 3 correct”. And then vary and permute the questions so nobody uses 20 accounts to do the same task and not separately answer the questions.
But that’s expensive and time-consuming, and unless the paper specifies it, one should assume they did the simpler/cheaper option of just paying people to answer.
Monetary incentives raise solution rates a little, but not that much. Lex Borghans and co-authors manipulate small incentives and they do almost nothing. Ben Enke and co-authors offers a full month’s salary to Nairobi based undergrads and finds a 13% percentage point increase in solution rates.
I’m not sure how our manipulations would interact with monetary incentives. But I’d like to know!