No, sorry, that should say confidences everywhere, not probabilities. I had written it out incorrectly and then edited it, but I missed that one. Fixed now.
The confidence level compares the answer to other answers Watson’s given in the past, based on how much the answer is supported by the evidence Watson has and uses. All the answers are generated and scored in parallel. It’s not a comparison among the answers generated for a specific question, so it shouldn’t necessarily add up to 100.
Quote from Chris Welty at last night’s panel: “When [Watson] says ‘this is my answer, 50% sure,’ half the time he’s right about that, and half the time he’s wrong. When he says 80%, 20% of the time he’s wrong.”
No, sorry, that should say confidences everywhere, not probabilities. I had written it out incorrectly and then edited it, but I missed that one. Fixed now.
What I meant was “for the top three answers, the confidences would sometimes sum to > 100, so how does that work?”
Is the procedure defined as well calibrated only for the top answer, or is there something I’m missing?
The confidence level compares the answer to other answers Watson’s given in the past, based on how much the answer is supported by the evidence Watson has and uses. All the answers are generated and scored in parallel. It’s not a comparison among the answers generated for a specific question, so it shouldn’t necessarily add up to 100.
Quote from Chris Welty at last night’s panel: “When [Watson] says ‘this is my answer, 50% sure,’ half the time he’s right about that, and half the time he’s wrong. When he says 80%, 20% of the time he’s wrong.”
Ah, thanks.