Intransitive preferences can be found from a series of binary choices, but if you force a ranking among the full set, you won’t have intransitive preferences (i.e., you can write out a gradient). This also means the elicitation procedure affects your inferences about the vectors. It would seem that circular preferences “fit,” but really they could just be fitting the (preferences | elicitation method) rather than “unconditional” (“core” + “irrationality,” whatever irrationality means) preferences. Preferences are also not independent of “irrelevant” alternatives as perceived attribute levels are evaluated contextually (that’s necessarily irrational?).
One implication I see here is that 0 vectors are points with no inclination to switch or having “no desire.” These would be useful model falsification points (e.g., Figure 7 implies that people don’t care about sportiness at all conditional on weight being “right”). But they would also only seem to correspond to ideal points or “ideal configuration” points. Without data on what the agent wants and only on what they are being offered (“I want a sporty car, but not too sporty; Car A is closest, but still not quite right, too bad”), you’ll be fitting the wrong hill to run up.
One problem with getting peoples’ entire ranking at once is that we’re cognitively incapable of ranking all states of the universe, so some approximation has to be used.
Your point about the elicitation method is interesting. In some sense, the problem is utterly inescapable, because “what do you think about A?” is literally a different elicitation than “what do you think about B?”, prompting the listener to think of different rating criteria.
Yeah, I think the points about elicitation methods and about the influence of irrelevant alternatives are important, and I don’t really have a great answer/solution. But I can say that this sounds quite related to some problems Stuart Armstrong has posted about in quite a few places, and that he seems to have some useful ideas for. E.g. in the “”Reasonable” situations” section of this post.
Charlie Steiner, right, it’s not doable for, say, all products on (or could be on) the market, but it is certainly doable among the products in a person’s consideration set. If we posit that they would make a choice among 4, then eliciting binary preferences might—but also might not—faithfully reflect how preferences look in the 4 set. So to MichaelA’s point, if preferences are context-dependent, then you need to identify appropriate contexts, or reasonable situations.
Context-dependent preferences present a big problem because “true” context-less preferences...maybe don’t exist. At the very least, we can make sure we’re eliciting preferences in an ecologically-valid way.
Binary choices are useful, but when they lead to inconsistencies, one should wonder whether it’s because preferences are inconsistent or whether it’s an elicitation thing. If people really would choose between A and B and not consider C or D, then ranking A and B is the relevant question. If people would consider A, B, C, and D (or at least pick between A and B in the context of C and D) then ranking all four (or at least ranking A and B in the context of C and D) is the relevant question.
Without data on what the agent wants and only on what they are being offered (“I want a sporty car, but not too sporty; Car A is closest, but still not quite right, too bad”), you’ll be fitting the wrong hill to run up.
Very neat post.
Intransitive preferences can be found from a series of binary choices, but if you force a ranking among the full set, you won’t have intransitive preferences (i.e., you can write out a gradient). This also means the elicitation procedure affects your inferences about the vectors. It would seem that circular preferences “fit,” but really they could just be fitting the (preferences | elicitation method) rather than “unconditional” (“core” + “irrationality,” whatever irrationality means) preferences. Preferences are also not independent of “irrelevant” alternatives as perceived attribute levels are evaluated contextually (that’s necessarily irrational?).
One implication I see here is that 0 vectors are points with no inclination to switch or having “no desire.” These would be useful model falsification points (e.g., Figure 7 implies that people don’t care about sportiness at all conditional on weight being “right”). But they would also only seem to correspond to ideal points or “ideal configuration” points. Without data on what the agent wants and only on what they are being offered (“I want a sporty car, but not too sporty; Car A is closest, but still not quite right, too bad”), you’ll be fitting the wrong hill to run up.
One problem with getting peoples’ entire ranking at once is that we’re cognitively incapable of ranking all states of the universe, so some approximation has to be used.
Your point about the elicitation method is interesting. In some sense, the problem is utterly inescapable, because “what do you think about A?” is literally a different elicitation than “what do you think about B?”, prompting the listener to think of different rating criteria.
Yeah, I think the points about elicitation methods and about the influence of irrelevant alternatives are important, and I don’t really have a great answer/solution. But I can say that this sounds quite related to some problems Stuart Armstrong has posted about in quite a few places, and that he seems to have some useful ideas for. E.g. in the “”Reasonable” situations” section of this post.
Charlie Steiner, right, it’s not doable for, say, all products on (or could be on) the market, but it is certainly doable among the products in a person’s consideration set. If we posit that they would make a choice among 4, then eliciting binary preferences might—but also might not—faithfully reflect how preferences look in the 4 set. So to MichaelA’s point, if preferences are context-dependent, then you need to identify appropriate contexts, or reasonable situations.
Context-dependent preferences present a big problem because “true” context-less preferences...maybe don’t exist. At the very least, we can make sure we’re eliciting preferences in an ecologically-valid way.
Binary choices are useful, but when they lead to inconsistencies, one should wonder whether it’s because preferences are inconsistent or whether it’s an elicitation thing. If people really would choose between A and B and not consider C or D, then ranking A and B is the relevant question. If people would consider A, B, C, and D (or at least pick between A and B in the context of C and D) then ranking all four (or at least ranking A and B in the context of C and D) is the relevant question.
Perhaps the missing variable is cost.