Well, then I think you haven’t put enough thought into how the system might be gamed (as it would be in practice). With your initial naive version, there would be an incentive to weigh in only on decisions that are slam-dunks and on decisions that you personally have a stake in, using the first to “buy” credibility that you “spend” on the other. Because of this, difficult decisions would be dominated by people with ulterior motives.
Now, of course there can be fixes for this, but it serves to illustrate that your system probably won’t be perfect fresh out of the box. Again, I think it would be an improvement on a system that doesn’t even track people’s records, but I don’t share your total zeal.
Also, people’s decision making abilities change over time. What I did right or wrong 5 years ago is not as important as what I did right or wrong one week ago. So, the influence of past decision scores should diminish as time passes. Exponential decay of importance, maybe? Another way you could do it is use a rating number for each player analogue to those used e.g. in chess (ELO).
Bah. Implementation details. A good algorithm uses the size of a voter’s record as well as the quality, and a great one might specifically give credit for being right when everyone else was wrong.
What we need is a theorem that tells us what the optimal algorithm is. Otherwise, there’s no evidence that this good algorithm of yours exists, and you can hide all sorts of implementation details under it.
With a restricted domain and certain assumptions, I do think I’ve found the Best System Ever. The first post was because I’m not confident.
Well, then I think you haven’t put enough thought into how the system might be gamed (as it would be in practice). With your initial naive version, there would be an incentive to weigh in only on decisions that are slam-dunks and on decisions that you personally have a stake in, using the first to “buy” credibility that you “spend” on the other. Because of this, difficult decisions would be dominated by people with ulterior motives.
Now, of course there can be fixes for this, but it serves to illustrate that your system probably won’t be perfect fresh out of the box. Again, I think it would be an improvement on a system that doesn’t even track people’s records, but I don’t share your total zeal.
Also, people’s decision making abilities change over time. What I did right or wrong 5 years ago is not as important as what I did right or wrong one week ago. So, the influence of past decision scores should diminish as time passes. Exponential decay of importance, maybe? Another way you could do it is use a rating number for each player analogue to those used e.g. in chess (ELO).
Bah. Implementation details. A good algorithm uses the size of a voter’s record as well as the quality, and a great one might specifically give credit for being right when everyone else was wrong.
What we need is a theorem that tells us what the optimal algorithm is. Otherwise, there’s no evidence that this good algorithm of yours exists, and you can hide all sorts of implementation details under it.
Good point.
(Open face, insert palm)