Not noise—I really mean a variable. For example, say you want to tax cigarettes, and someone else wants to ban cigarettes. To find out who knows the other person’s position better you trade places and try to fool some large group of their peers. The taxer chats with some banners for a while, and the banner chats with some taxers for a while. Then the two groups write down how knowledgeable they thought people were.
But what if taxers tend to be introverts, and banners tend to be police officers who are masters at interrogation? Or, more plausibly, what if there are group-specific things that will impact peoples’ judgement of your knowledge other than the substance of the issue? There are all sorts of things that could cause groups to grade transplants differently, and these factors are left uncontrolled if you just do the swap as suggested.
Not noise—I really mean a variable. For example, say you want to tax cigarettes, and someone else wants to ban cigarettes. To find out who knows the other person’s position better you trade places and try to fool some large group of their peers. The taxer chats with some banners for a while, and the banner chats with some taxers for a while. Then the two groups write down how knowledgeable they thought people were.
But what if taxers tend to be introverts, and banners tend to be police officers who are masters at interrogation? Or, more plausibly, what if there are group-specific things that will impact peoples’ judgement of your knowledge other than the substance of the issue? There are all sorts of things that could cause groups to grade transplants differently, and these factors are left uncontrolled if you just do the swap as suggested.
Ah, OK then. (Selection bias.)