tl;dr: If this post included a section discussing push-poll concerns and advocating (at least) caution and (preferably) a policy that’d be robust against human foibles, I’d be interested in having this post in the 2019 Review Book.
I think this is an interesting idea that should likely get experimented with.
A thing I was worried about when this first came out, and still worried about, is the blurriness between “survey as tool to gather data” and “survey as tool to cause action in the respondent.”
Some commenters said “this seems like push-polling, isn’t that bad?”.
I think a carefully executed version of this wouldn’t be push-polling. I’m not 100% sure whether the example here counts or not. It doesn’t prescribe an action. But it does prescribe a frame. There’s a spectrum of push-polliness, and I say the example here is non-zero push-poll-y, even if not maximally.
I don’t think it’s intrinsically bad to be push-poll-y, but I think it’s useful to draw distinctions between surveys designed for gathering information vs shaping actions. If a tool like this started seeing common use, we’d a) start seeing some variants of it that were push-poll-y (because humans aren’t always careful), and b) I think people receiving the survey would be anxious about it being push-poll-y, which would compromise their trust in future surveys on the same topic or from the same institutions. This may be unfair, but it’s the outcome I predict.
One commenter suggested flagging such a survey as a “coordination survey” rather than a “information gathering survey.” This is somewhat unfair to hypothetical survey-makers who were very careful to not be a push poll. But, I think it is probably the best equilibrium.
(Flagging polls as “coordination surveys” will probably come with some consequences, where people who are already mistrustful of your cause and don’t want to coordinate on it are more likely to avoid the survey in the first place. But the alternative is manipulating them into taking a survey that wasn’t quite what they thought it was. I think that is worse)
tl;dr: If this post included a section discussing push-poll concerns and advocating (at least) caution and (preferably) a policy that’d be robust against human foibles, I’d be interested in having this post in the 2019 Review Book.
I think this is an interesting idea that should likely get experimented with.
A thing I was worried about when this first came out, and still worried about, is the blurriness between “survey as tool to gather data” and “survey as tool to cause action in the respondent.”
Some commenters said “this seems like push-polling, isn’t that bad?”.
I think a carefully executed version of this wouldn’t be push-polling. I’m not 100% sure whether the example here counts or not. It doesn’t prescribe an action. But it does prescribe a frame. There’s a spectrum of push-polliness, and I say the example here is non-zero push-poll-y, even if not maximally.
I don’t think it’s intrinsically bad to be push-poll-y, but I think it’s useful to draw distinctions between surveys designed for gathering information vs shaping actions. If a tool like this started seeing common use, we’d a) start seeing some variants of it that were push-poll-y (because humans aren’t always careful), and b) I think people receiving the survey would be anxious about it being push-poll-y, which would compromise their trust in future surveys on the same topic or from the same institutions. This may be unfair, but it’s the outcome I predict.
One commenter suggested flagging such a survey as a “coordination survey” rather than a “information gathering survey.” This is somewhat unfair to hypothetical survey-makers who were very careful to not be a push poll. But, I think it is probably the best equilibrium.
(Flagging polls as “coordination surveys” will probably come with some consequences, where people who are already mistrustful of your cause and don’t want to coordinate on it are more likely to avoid the survey in the first place. But the alternative is manipulating them into taking a survey that wasn’t quite what they thought it was. I think that is worse)