It seems to me that the main problem with that article is that the survey questions were badly designed. The writer claims that the questions “tested people’s real-world understanding of basic economic principles” – but in fact they merely tested people’s willingness to interpret vague or misleading questions (with nothing at stake...hardly “real-world”) as the experimenter intended.
Given that ambiguity it is hardly surprising that respondents chose to interpret the questions in a way that flatters their personal ideology. This does not demonstrate that the respondents are irrational in any way; if anyone is irrational it is the person who thinks that these surveys prove such a thing.
A few of the questions are listed with the “obviously false” answer in brackets:
a dollar means more to a poor person than it does to a rich person (disagree)
This doesn’t specify whether it is referring to an average rich/poor man, or as a general condition for all rich/poor men. If the former then it is obviously true, but if the latter then it is false. A highly motivated entrepreneur of a penny-pinching disposition might care more about a dollar than an ascetic monk, for example.
when two people complete a voluntary transaction, they both necessarily come away better off (agree)
when two people complete a voluntary transaction, it is necessarily the case that everyone else is unaffected by their transaction (agree)
The problem with these questions is that the possible responses were listed as “strongly agree”, “somewhat agree”, “somewhat disagree” etc. However, the only sensible answers to this question are obviously “yes” or “no” (or perhaps “unsure”). Since the list of possible responses suggest that in fact the answer is a matter of degree, it is easy for the respondent to believe that the question isn’t asking him a binary logical question but actually probing his degree of commitment to the idea of free markets or some such thing.
drug prohibition fails to reduce people’s access to drugs (agree)
gun-control laws fail to reduce people’s access to guns (agree)
The same applies to these question – why aren’t the possible answers just “yes”, “no” and “unsure”? Respondents will feel, given the graduated nature of the possible answers suggested, that what is really being probed is the question (which is after all far more interesting and salient) whether they feel that drug prohibition and gun-control laws are successful policies. Asking whether these policies reduce access to drugs and guns at all is so trivial that, given the graduated set of possible responses, it is almost sensible that respondents should answer as though the question was whether they feel that these policies are successful.
Furthermore there is some ambiguity is the sense in which “reduce access” is intended. In one sense gun-control laws reduce access, because they make it harder to get a gun. On the other hand it could be plausibly argued that they do not actually prevent a determined person from acquiring the guns he wants, therefore access to guns hasn’t been reduced per se but merely made more time-consuming or dangerous. It depends whether access refers to ease of getting a gun, or plain ability to get a gun (or drugs) given a high motivation.
overall, the standard of living is higher today than it was 30 years ago (disagree)
Is this really blatantly true? Here is an article that I do not necessarily endorse, but which demonstrates that it is rather presumptuous to pronounce that “disagree” is obviously wrong in this case. Here is Elizabeth Warren on the “two-income trap”. And of course there is the small matter of the financial crisis.
Also, the question fails to mention whether “overall” refers to America in general or to the world in general, which might have some bearing on the answer.
Third World workers working for American companies overseas are being exploited (agree)
This would seem to hinge on the definition of “exploited”. And the question doesn’t specify who is exploiting the Third Worlders: the companies in question, or the capitalist First World system in general. Perhaps a socialist might argue that they are being exploited because we haven’t compensated them properly for the sins of colonialism, therefore putting them in the position where they have to work in sweatshops to make ends meet. Again it is not inevitable that any intelligent individual would accept that this statement is blatantly false, even after having that “fact” pointed out to him.
This would seem to hinge on the definition of “exploited”. And the question doesn’t specify who is exploiting the Third Worlders: the companies in question, or the capitalist First World system in general. Perhaps a socialist might argue that they are being exploited because we haven’t compensated them properly for the sins of colonialism, therefore putting them in the position where they have to work in sweatshops to make ends meet. Again it is not inevitable that any intelligent individual would accept that this statement is blatantly false, even after having that “fact” pointed out to him.
More than that, a socialist would almost certainly argue that they are being exploited by the landowner, by the recipient of any fee they have to pay (for instance, for intellectual property) their own government if they pay taxes, and so on. The socialist definition of exploitation is extremely broad but roughly isomorphic to rent. It’s also to my knowledge the only remotely rigorous definition of exploitation that would make sense in that context. So the question is pretty much explicitly asking “are you a socialist” and taking yes as being wrong about economics. Since the author’s of the study disagree with socialists about economics that seems entirely fair, though obviously as an argument that socialists don’t understand economics it’s circular. Still it would be clearer if they said “demonstrably being exploited”, but I think they are assuming that people who think exploited is vague default to no.
In the context of the study, the problem is that the wrongness of certain answers in the survey is supposed to be attributable only to political bias. I am not a socialist but I think it’s fair to say that the socialist view of economics, as far as economic facts are concerned, is not merely a bias. It may be wrong, but it should be credited at least as a mistaken school of thought rather than having all of its conclusions attributed to “myside bias”.
Although as I pointed out there are other problems with the survey, in this case the investigators should have limited themselves to questions that after correction, practically any intelligent person would agree upon—i.e. as close as possible to tautology whilst retaining the possibility for political bias to affect the results.
I can’t think of a good way to reword that particular survey question, so I’d probably scrap it. On the other hand:
a dollar means more to a poor person than it does to a rich person (disagree)
Ceteris paribus an average poor person, in comparison to an average rich person, is likely to be more motivated to perform some task (of which he is presumed capable) by the promise of a given monetary reward.
drug prohibition fails to reduce people’s access to drugs
Drug prohibition doesn’t increase the difficulty or inconvenience experienced by an average individual in obtaining hard drugs whatsoever
Answer Agree/Disagree/Unsure
I would expect “myside bias” to be less evident if the survey questions were like this (i.e. not very open to interpretation).
It seems to me that the main problem with that article is that the survey questions were badly designed. The writer claims that the questions “tested people’s real-world understanding of basic economic principles” – but in fact they merely tested people’s willingness to interpret vague or misleading questions (with nothing at stake...hardly “real-world”) as the experimenter intended.
Given that ambiguity it is hardly surprising that respondents chose to interpret the questions in a way that flatters their personal ideology. This does not demonstrate that the respondents are irrational in any way; if anyone is irrational it is the person who thinks that these surveys prove such a thing.
A few of the questions are listed with the “obviously false” answer in brackets:
This doesn’t specify whether it is referring to an average rich/poor man, or as a general condition for all rich/poor men. If the former then it is obviously true, but if the latter then it is false. A highly motivated entrepreneur of a penny-pinching disposition might care more about a dollar than an ascetic monk, for example.
The problem with these questions is that the possible responses were listed as “strongly agree”, “somewhat agree”, “somewhat disagree” etc. However, the only sensible answers to this question are obviously “yes” or “no” (or perhaps “unsure”). Since the list of possible responses suggest that in fact the answer is a matter of degree, it is easy for the respondent to believe that the question isn’t asking him a binary logical question but actually probing his degree of commitment to the idea of free markets or some such thing.
The same applies to these question – why aren’t the possible answers just “yes”, “no” and “unsure”? Respondents will feel, given the graduated nature of the possible answers suggested, that what is really being probed is the question (which is after all far more interesting and salient) whether they feel that drug prohibition and gun-control laws are successful policies. Asking whether these policies reduce access to drugs and guns at all is so trivial that, given the graduated set of possible responses, it is almost sensible that respondents should answer as though the question was whether they feel that these policies are successful.
Furthermore there is some ambiguity is the sense in which “reduce access” is intended. In one sense gun-control laws reduce access, because they make it harder to get a gun. On the other hand it could be plausibly argued that they do not actually prevent a determined person from acquiring the guns he wants, therefore access to guns hasn’t been reduced per se but merely made more time-consuming or dangerous. It depends whether access refers to ease of getting a gun, or plain ability to get a gun (or drugs) given a high motivation.
Is this really blatantly true? Here is an article that I do not necessarily endorse, but which demonstrates that it is rather presumptuous to pronounce that “disagree” is obviously wrong in this case. Here is Elizabeth Warren on the “two-income trap”. And of course there is the small matter of the financial crisis.
Also, the question fails to mention whether “overall” refers to America in general or to the world in general, which might have some bearing on the answer.
This would seem to hinge on the definition of “exploited”. And the question doesn’t specify who is exploiting the Third Worlders: the companies in question, or the capitalist First World system in general. Perhaps a socialist might argue that they are being exploited because we haven’t compensated them properly for the sins of colonialism, therefore putting them in the position where they have to work in sweatshops to make ends meet. Again it is not inevitable that any intelligent individual would accept that this statement is blatantly false, even after having that “fact” pointed out to him.
I think most of your assessments are accurate.
More than that, a socialist would almost certainly argue that they are being exploited by the landowner, by the recipient of any fee they have to pay (for instance, for intellectual property) their own government if they pay taxes, and so on. The socialist definition of exploitation is extremely broad but roughly isomorphic to rent. It’s also to my knowledge the only remotely rigorous definition of exploitation that would make sense in that context. So the question is pretty much explicitly asking “are you a socialist” and taking yes as being wrong about economics. Since the author’s of the study disagree with socialists about economics that seems entirely fair, though obviously as an argument that socialists don’t understand economics it’s circular. Still it would be clearer if they said “demonstrably being exploited”, but I think they are assuming that people who think exploited is vague default to no.
Thanks for the elaboration, which I endorse.
In the context of the study, the problem is that the wrongness of certain answers in the survey is supposed to be attributable only to political bias. I am not a socialist but I think it’s fair to say that the socialist view of economics, as far as economic facts are concerned, is not merely a bias. It may be wrong, but it should be credited at least as a mistaken school of thought rather than having all of its conclusions attributed to “myside bias”.
Although as I pointed out there are other problems with the survey, in this case the investigators should have limited themselves to questions that after correction, practically any intelligent person would agree upon—i.e. as close as possible to tautology whilst retaining the possibility for political bias to affect the results.
I can’t think of a good way to reword that particular survey question, so I’d probably scrap it. On the other hand:
Ceteris paribus an average poor person, in comparison to an average rich person, is likely to be more motivated to perform some task (of which he is presumed capable) by the promise of a given monetary reward.
Drug prohibition doesn’t increase the difficulty or inconvenience experienced by an average individual in obtaining hard drugs whatsoever
Answer Agree/Disagree/Unsure
I would expect “myside bias” to be less evident if the survey questions were like this (i.e. not very open to interpretation).