Personally, I see this question as being an empirical question. Which method works best?
In the cases I care about, both averaging log odds and taking a median far outperform taking a mean. (Fwiw Metaculus agrees that it’s a very safe bet too)
In contrast, there are no conditions under which average log odds is the correct thing to, because violating additivity of disjoint events is never the correct thing to do (see previous paragraph).
Your example about additivity of disjoint events is somewhat contrived. Averaging log-odds respects the probability for a given event summing to 1, but if you add some additional structure it might not make sense, I agree.
Averaging log-odds is exactly a Bayesian update, so presumably you’d accept there are some conditions under which average log odds is the correct thing to do...
Your example about additivity of disjoint events is somewhat contrived. Averaging log-odds respects the probability for a given event summing to 1, but if you add some additional structure it might not make sense, I agree.
Contrived how? What additional structure do you imagine I added? In what sense do you claim that averaging log odds preserves additivity of probability for disjoint events in the face of an example showing that the straightforward interpretation of this claim is false?
Averaging log-odds is exactly a Bayesian update
It isn’t; you can tell because additivity of probability for disjoint events continues to hold after Bayesian updates. [Edit: Perhaps a better explanation for why it isn’t a Bayesian update is that it isn’t even the same type signature as a Bayesian update. A Bayesian update takes a probability distribution and some evidence, and returns a probability distribution. Averaging log-odds takes some finite set of probabilities, and returns a probability]. I’m curious what led you to believe this, though.
I think it would be perhaps helpful to link to a few people advocating averaging log-odds rather than averaging probabilities, eg:
When pooling forecasts, use the geometric mean of the odds
My current best guess on how to aggregate forecasts
Personally, I see this question as being an empirical question. Which method works best?
In the cases I care about, both averaging log odds and taking a median far outperform taking a mean. (Fwiw Metaculus agrees that it’s a very safe bet too)
Your example about additivity of disjoint events is somewhat contrived. Averaging log-odds respects the probability for a given event summing to 1, but if you add some additional structure it might not make sense, I agree.
Averaging log-odds is exactly a Bayesian update, so presumably you’d accept there are some conditions under which average log odds is the correct thing to do...
Thanks for the links!
Contrived how? What additional structure do you imagine I added? In what sense do you claim that averaging log odds preserves additivity of probability for disjoint events in the face of an example showing that the straightforward interpretation of this claim is false?
It isn’t; you can tell because additivity of probability for disjoint events continues to hold after Bayesian updates. [Edit: Perhaps a better explanation for why it isn’t a Bayesian update is that it isn’t even the same type signature as a Bayesian update. A Bayesian update takes a probability distribution and some evidence, and returns a probability distribution. Averaging log-odds takes some finite set of probabilities, and returns a probability]. I’m curious what led you to believe this, though.