So, let’s say you want a scoring rule with two properties.
You want it to be local: that is to say, all that matters is the probability you assigned to the actual outcome. This is in contrast to rules like the quadratic scoring rule, where your score is different depending on how the outcomes that didn’t happen are grouped. Based on this assumption, I’m going to write the scoring rule as S(p), where S(p) is the score you get when you assign a probability p to the true outcome.
You also want it to play nicely with combining separate events. That is to say, if you estimate 10% of it being cloudy when it actually is, and 10% of it being warm outside when it actually is, you want your score to be the same as if you had assigned 1% to the correct proposition that it is warm and cloudy outside. More succinctly:
S(p)+S(q)=S(pq).
If you add in the additional caveat that some scores are not 0, then you are forced by the above statement to a logarithmic scoring rule. Interestingly, you don’t need to include the requirement that it be a proper scoring rule, although the logarithmic scoring rule is proper.
So, let’s say you want a scoring rule with two properties.
You want it to be local: that is to say, all that matters is the probability you assigned to the actual outcome. This is in contrast to rules like the quadratic scoring rule, where your score is different depending on how the outcomes that didn’t happen are grouped. Based on this assumption, I’m going to write the scoring rule as S(p), where S(p) is the score you get when you assign a probability p to the true outcome.
You also want it to play nicely with combining separate events. That is to say, if you estimate 10% of it being cloudy when it actually is, and 10% of it being warm outside when it actually is, you want your score to be the same as if you had assigned 1% to the correct proposition that it is warm and cloudy outside. More succinctly: S(p)+S(q)=S(pq).
If you add in the additional caveat that some scores are not 0, then you are forced by the above statement to a logarithmic scoring rule. Interestingly, you don’t need to include the requirement that it be a proper scoring rule, although the logarithmic scoring rule is proper.