but predicted that it was instead about sensitivity to subtle changes in the wording of questions.
If I try this again next year I’m inclined to keep the wording the same instead of trying to be subtle.
Regarding the dutch book numbers: it seems like, for each of the individual-question presentations of that data, you removed the outliers. When performing the dutch book calculations, however, it seems like you keep the outliers in.
Yep. Well, in the individual reports I reported the version with the outliers, and then sometimes did another pass without outliers. I kept all of the entries that answered all the questions for the dutch book calculations, even if they were outliers. I think this is the correct move: if someone’s valuations are wild outliers from everyone elses but in a way that multiplies out and gets them back to a 1:1 ratio, then being an outlier isn’t a problem.
(Imagine someone who values a laptop at one million bikes, and a bike at equal to one car, and a car at one millionth of a of a laptop. They’re almost certainly a wild outlier, and I’m confused as heck, but they are consistent in their values!)
I think your reasoning-as-stated there is true and I’m glad that you showed the full data. I suggested removing outliers for dutch book calculations because I suspected that the people who were wild outliers on at least one of their answers were more likely to be wild outliers on their ability to resist dutch books; I predict that the thing that causes someone to say they value a laptop at one million bikes is pretty often just going to be “they’re unusually bad at assigning numeric values to things.”
The actual origin of my confusion was “huh, those dutch book numbers look really high relative to my expectations, this reminds me of earlier in the post when the other outliers made numbers really high.”
I’d be interested to see the outlier-less numbers here, but I respect if you don’t have the spoons for that given that the designated census processing time is already over.
If I try this again next year I’m inclined to keep the wording the same instead of trying to be subtle.
Yep. Well, in the individual reports I reported the version with the outliers, and then sometimes did another pass without outliers. I kept all of the entries that answered all the questions for the dutch book calculations, even if they were outliers. I think this is the correct move: if someone’s valuations are wild outliers from everyone elses but in a way that multiplies out and gets them back to a 1:1 ratio, then being an outlier isn’t a problem.
(Imagine someone who values a laptop at one million bikes, and a bike at equal to one car, and a car at one millionth of a of a laptop. They’re almost certainly a wild outlier, and I’m confused as heck, but they are consistent in their values!)
I think your reasoning-as-stated there is true and I’m glad that you showed the full data. I suggested removing outliers for dutch book calculations because I suspected that the people who were wild outliers on at least one of their answers were more likely to be wild outliers on their ability to resist dutch books; I predict that the thing that causes someone to say they value a laptop at one million bikes is pretty often just going to be “they’re unusually bad at assigning numeric values to things.”
The actual origin of my confusion was “huh, those dutch book numbers look really high relative to my expectations, this reminds me of earlier in the post when the other outliers made numbers really high.”
I’d be interested to see the outlier-less numbers here, but I respect if you don’t have the spoons for that given that the designated census processing time is already over.