It’s not a Bayesian method, but you could use Randall Munroe’s algorithm, probably adding on the assumption of transitivity for each participant except where it is explicitly violated. The way that works is:
Step 1: you say that participant 1 has A>B if 1) they explicitly ranked A>B, or if 2) they have a transitive path giving A>B and do not have a transitive path giving B>A. Repeat for each pair of alternatives for each participant. (note that a participant may have A=B)
Step 2: you say that the group has A>B if more participants have A>B than have B>A. Repeat for each pair of alternatives. (note that the group may have A=B)
Step 3: you count up how many alternatives A is preferred to by the group, and divide by that number plus the number of objects that are preferred to A. This is A’s score. Repeat for each alternative.
If you only care about which object is most likely to be #1, and not any other ranks, then another method is to look for each participant only at the objects that are undefeated. Each object that was never rejected by participant 1 would get a score of 1/(# of undefeated objects), and the other objects would get a score of 0. Then you just sum the scores across participants. Or, you might want not want to give the same score to an object that the participant never saw as you do to one that they preferred three times, so you could weight by number of times selected; perhaps give the object a score of (1 + # of objects it was preferred to)/(sum numerators), where you divide by the sum to normalize each participant’s scores to add to one.
It’s not a Bayesian method, but you could use Randall Munroe’s algorithm, probably adding on the assumption of transitivity for each participant except where it is explicitly violated. The way that works is:
Step 1: you say that participant 1 has A>B if 1) they explicitly ranked A>B, or if 2) they have a transitive path giving A>B and do not have a transitive path giving B>A. Repeat for each pair of alternatives for each participant. (note that a participant may have A=B)
Step 2: you say that the group has A>B if more participants have A>B than have B>A. Repeat for each pair of alternatives. (note that the group may have A=B)
Step 3: you count up how many alternatives A is preferred to by the group, and divide by that number plus the number of objects that are preferred to A. This is A’s score. Repeat for each alternative.
If you only care about which object is most likely to be #1, and not any other ranks, then another method is to look for each participant only at the objects that are undefeated. Each object that was never rejected by participant 1 would get a score of 1/(# of undefeated objects), and the other objects would get a score of 0. Then you just sum the scores across participants. Or, you might want not want to give the same score to an object that the participant never saw as you do to one that they preferred three times, so you could weight by number of times selected; perhaps give the object a score of (1 + # of objects it was preferred to)/(sum numerators), where you divide by the sum to normalize each participant’s scores to add to one.
Wow, that’s so simple it could possibly work!
Many thanks—this is most likely what I’ll go with. I appreciate your help. :-)