Try to think about this in terms of expected value. On your specific example, they do score more, but this is probabilistic thinking, so we want to think about it in terms of the long run trend.
Suppose we no longer know what the answer is, and you are genuinely 50⁄50 on it being either A or B. This is what you truly believe, you don’t think there’s a chance in hell it’s C. If you sit there and ask yourself, “Maybe I should do a 50-25-25 split, just in case”, you’re going to immediately realize “Wait, that’s moronic. I’m throwing away 25% of my points on something I am certain is wrong. This is like betting on a 3-legged horse.”
Now let’s say you do a hundred of these questions, and most of your 50-50s come up correct-ish as one or the other. Your opponent consistently does 50-25-25s, and so they end up more wrong than you overall, because half the time the answer lands on one of their two 25s, not their single 50.
It’s not a game of being more correct, it’s a game of being less wrong.
I think all of this is also true of a scoring rule based on only the probability placed on the correct answer?
In the end you’d still expect to win but this takes longer (requires more questions) under a rule which includes probabilities on incorrect answers—it’s just adding noise to the results.
Try to think about this in terms of expected value. On your specific example, they do score more, but this is probabilistic thinking, so we want to think about it in terms of the long run trend.
Suppose we no longer know what the answer is, and you are genuinely 50⁄50 on it being either A or B. This is what you truly believe, you don’t think there’s a chance in hell it’s C. If you sit there and ask yourself, “Maybe I should do a 50-25-25 split, just in case”, you’re going to immediately realize “Wait, that’s moronic. I’m throwing away 25% of my points on something I am certain is wrong. This is like betting on a 3-legged horse.”
Now let’s say you do a hundred of these questions, and most of your 50-50s come up correct-ish as one or the other. Your opponent consistently does 50-25-25s, and so they end up more wrong than you overall, because half the time the answer lands on one of their two 25s, not their single 50.
It’s not a game of being more correct, it’s a game of being less wrong.
I think all of this is also true of a scoring rule based on only the probability placed on the correct answer?
In the end you’d still expect to win but this takes longer (requires more questions) under a rule which includes probabilities on incorrect answers—it’s just adding noise to the results.