I wonder how many people cooperated only (or in part) because they knew the results would be correlated with their (political) views, and they wanted their “tribe”/community/group/etc. to look good.
I don’t think the responses of people here would be so much affected by directly wanting to present their own social group as good. However (false) correlation between those two could happen just because of framing by other questions.
E.g. the answer to prisoner’s dilemma question might be affected by whether you’ve just answered “I’m associated with the political left” or whether you’ve just answered “I consider rational calculations to be the best way to solve issues”.
If that is the effect causing a false correlation, then adding the statment “these won’t be correlated” woudn’t do any good—in fact, it would only serve as a further activation for the person to enter the political-association frame.
This is a common problem with surveys that isn’t very easy to mitigate. Individually randomizing question order and analyzing differences in correlations based on presented question order helps a bit, but the problem still remains, and the sample size for any such difference-in-correlation analysis becomes increasingly small.
I don’t think the responses of people here would be so much affected by directly wanting to present their own social group as good. However (false) correlation between those two could happen just because of framing by other questions.
E.g. the answer to prisoner’s dilemma question might be affected by whether you’ve just answered “I’m associated with the political left” or whether you’ve just answered “I consider rational calculations to be the best way to solve issues”.
If that is the effect causing a false correlation, then adding the statment “these won’t be correlated” woudn’t do any good—in fact, it would only serve as a further activation for the person to enter the political-association frame.
This is a common problem with surveys that isn’t very easy to mitigate. Individually randomizing question order and analyzing differences in correlations based on presented question order helps a bit, but the problem still remains, and the sample size for any such difference-in-correlation analysis becomes increasingly small.