The primary property you want to maintain with a scoring rule is that the best probability to provide is your true probability. I know that the Bayes score generalizes to multiple choice questions, which implies to me that it most likely works with a multiplicity for wrong answers, so long as the multiplicity is close to the actual multiplicity.
I think the primary property you want to maintain is that it’s best to provide the answer you consider most likely, otherwise it’s best to say ‘sdfkhasflk’ − 0% to all of them you aren’t certain of.
Multiple choice would making the scoring clearer, but that constraint could well make the calibration easier.
The primary property you want to maintain with a scoring rule is that the best probability to provide is your true probability. I know that the Bayes score generalizes to multiple choice questions, which implies to me that it most likely works with a multiplicity for wrong answers, so long as the multiplicity is close to the actual multiplicity.
I think the primary property you want to maintain is that it’s best to provide the answer you consider most likely, otherwise it’s best to say ‘sdfkhasflk’ − 0% to all of them you aren’t certain of.
Multiple choice would making the scoring clearer, but that constraint could well make the calibration easier.