I don’t exactly disagree, but I think this focuses on the wrong thing. It’s not about slowing down in your calculations, nor about updating by less for each data point (what I initially thought “slow” meant). Calculate your updates as fast as you like, as long as you do it correctly. Bayes Theorem gives us a numerical approach to updates—if you’re making bigger jumps than are justified by the evidence, that’s an error. If you’re making smaller jumps, that’s also an error.
Likewise, assuming that the truth is in your hypothesis set is a mistake. You should include “other” in your modeling at this level. More importantly, you should recognize that coinflip outcomes provide _zero_ evidence about the relative likelihood between cheating and psychic (and “other”). Each successful prediction _does_ provide evidence against luck and toward those other possibilities.
Getting better at updating is a skill, and “go slow” is probably good advice when practicing (per the standard saying “Slow is smooth and smooth is fast”), but doesn’t generalize. Time is a limited resource and you’re spending valuable seconds here that you’d rather be doing something else.
I should clarify that what I mean by “slow” was supposed to be in the cognitive / Kahneman sense. In most cases, as I said, “if you don’t need to make any decisions, at least file everything away and decide that it’s unclear.” Instead, what I see people do is jump to updating / grabbing a hypothesis, acting “fast.” The failure modes that this can cause will include over or under-updating, ignoring “missing” hypotheses and not thinking of alternative explanations, narrowing the hypotheses too early, not updating marginally, etc.
Given that, I think I agree on all of the substantive points. However, I will note that including an explicit “other” category is tricky, but critical. The problem is that given such a category, over time it’s more plausible than anything else. It turns into the equivalent of “the witch down the road did it,” which is super-parsimonious and can explain anything.
And slowness in the sense you thought I was talking about is equivalent to lowering the weight on evidence, or to having a higher weight on priors. Strength of priors and how much to weight evidence are good questions to discuss, and can be tricky, but weren’t my point in this post.
I don’t exactly disagree, but I think this focuses on the wrong thing. It’s not about slowing down in your calculations, nor about updating by less for each data point (what I initially thought “slow” meant). Calculate your updates as fast as you like, as long as you do it correctly. Bayes Theorem gives us a numerical approach to updates—if you’re making bigger jumps than are justified by the evidence, that’s an error. If you’re making smaller jumps, that’s also an error.
Likewise, assuming that the truth is in your hypothesis set is a mistake. You should include “other” in your modeling at this level. More importantly, you should recognize that coinflip outcomes provide _zero_ evidence about the relative likelihood between cheating and psychic (and “other”). Each successful prediction _does_ provide evidence against luck and toward those other possibilities.
Getting better at updating is a skill, and “go slow” is probably good advice when practicing (per the standard saying “Slow is smooth and smooth is fast”), but doesn’t generalize. Time is a limited resource and you’re spending valuable seconds here that you’d rather be doing something else.
I should clarify that what I mean by “slow” was supposed to be in the cognitive / Kahneman sense. In most cases, as I said, “if you don’t need to make any decisions, at least file everything away and decide that it’s unclear.” Instead, what I see people do is jump to updating / grabbing a hypothesis, acting “fast.” The failure modes that this can cause will include over or under-updating, ignoring “missing” hypotheses and not thinking of alternative explanations, narrowing the hypotheses too early, not updating marginally, etc.
Given that, I think I agree on all of the substantive points. However, I will note that including an explicit “other” category is tricky, but critical. The problem is that given such a category, over time it’s more plausible than anything else. It turns into the equivalent of “the witch down the road did it,” which is super-parsimonious and can explain anything.
And slowness in the sense you thought I was talking about is equivalent to lowering the weight on evidence, or to having a higher weight on priors. Strength of priors and how much to weight evidence are good questions to discuss, and can be tricky, but weren’t my point in this post.