I think there’s a misunderstanding. You are supposed to ask the model for its probability estimate, not give your own probability estimate. The Brier score loss is based on the question-answer’s probabilities over possible answers, not the question-grader’s probabilities.
I think there’s a misunderstanding. You are supposed to ask the model for its probability estimate, not give your own probability estimate. The Brier score loss is based on the question-answer’s probabilities over possible answers, not the question-grader’s probabilities.