The correct method would probably involve cooking up a hypothetical bayesian judge that takes everyones estimates as evidence.
Weighting, in part, by the calibration questions?
I dunno how you would weight it. I think you’d want to have a maxentropy ‘fair’ judge at least for comparison.
Calibration questions are probably the least controversial way of weighting. Compare to, say, trying to weight using karma.
This might be an interesting thing to develop. A voting system backed up by solid bayes-math could be useful for more than just LW surveys.
It might be interesting to see what results are produced by several weighting approaches.
yeah. that’s what I was getting at with the maxentropy judge.
On further thought, I really should look into figuring this out. Maybe I’ll do some work on it and post a discussion post. This could be a great group rationality tool.
Weighting, in part, by the calibration questions?
I dunno how you would weight it. I think you’d want to have a maxentropy ‘fair’ judge at least for comparison.
Calibration questions are probably the least controversial way of weighting. Compare to, say, trying to weight using karma.
This might be an interesting thing to develop. A voting system backed up by solid bayes-math could be useful for more than just LW surveys.
It might be interesting to see what results are produced by several weighting approaches.
yeah. that’s what I was getting at with the maxentropy judge.
On further thought, I really should look into figuring this out. Maybe I’ll do some work on it and post a discussion post. This could be a great group rationality tool.