I am afraid that the HCH could be affected by Chinese whispers-like situation: that is, the accumulation of errors because each person has a little different understanding of the meaning of words, especially in the Mechanical Turk scenarios.
Yeah, this is a problem that needs to be addressed. It feels like in the Overseers Manual case you can counteract this by giving definitions/​examples of how you want questions to be interpreted, and in the Lookup Table case this can be addr by coordination within the team creating the lookup table
I am afraid that the HCH could be affected by Chinese whispers-like situation: that is, the accumulation of errors because each person has a little different understanding of the meaning of words, especially in the Mechanical Turk scenarios.
Yeah, this is a problem that needs to be addressed. It feels like in the Overseers Manual case you can counteract this by giving definitions/​examples of how you want questions to be interpreted, and in the Lookup Table case this can be addr by coordination within the team creating the lookup table