Despite the difficulty of exact Bayesian inference in complex mathematical models, the essence of Bayesian reasoning is frequently used in everyday life. One example has been immortalized in the words of Sherlock Holmes to his friend Dr. Watson: “How often have I said to you that when you have eliminated the impossible, whatever remains, however improbable, must be the truth?” (Arthur Conan Doyle, The Sign of Four, 1890, Ch. 6). This reasoning is actually a consequence of Bayesian belief updating, as expressed in Equation 4.4. Let me re-state it this way: “How often have I said to you that when p(D|θ_i ) = 0 for all i!=j, then, no matter how small the prior p(θ_j ) > 0 is, the posterior p(θ_j |D) must equal one.” Somehow it sounds better the way Holmes said it.
--Kruschke 2010, Doing Bayesian Data Analysis, pg56-57
When you have eliminated the impossible, whatever remains is often more improbable than your having made a mistake in one of your impossibility proofs.
It always irritates me slightly that Holmes says “whatever remains, however improbable, must be the truth”, when multiple incompatible hypotheses will remain.
My Holmes says, “When you have eliminated the possible, you must expand your conception of what is possible.”
You have an inequality symbol missing at the end of the quote (between i and j). That made it slightly difficult for me to parse it on my first read-through (“Why does it say ‘for all i, j’ when the only index in the expression is ‘i’?”).
I don’t know if you know, but just in case you (or someone else) don’t:
There is no inequality symbol on the computer keyboard, so he used a typical programmer’s inequality symbol which is ”!=”. So yes, it is not easily readable (i! is a bad combination...) but totally correct.
The way to handle that is whitespace: i != 0. (I once was teased by my tendency to put whitespace in computer code around all operators which would be spaced in typeset mathematical formulas.)
EDIT: I also use italics for variables, boldface for vectors, etc. when handwriting. Whenever I get a new pen I immediately check whether it’s practical to do boldface with it.
--Kruschke 2010, Doing Bayesian Data Analysis, pg56-57
-Steven Kaas (via)
It always irritates me slightly that Holmes says “whatever remains, however improbable, must be the truth”, when multiple incompatible hypotheses will remain.
My Holmes says, “When you have eliminated the possible, you must expand your conception of what is possible.”
You have an inequality symbol missing at the end of the quote (between i and j). That made it slightly difficult for me to parse it on my first read-through (“Why does it say ‘for all i, j’ when the only index in the expression is ‘i’?”).
I don’t know if you know, but just in case you (or someone else) don’t: There is no inequality symbol on the computer keyboard, so he used a typical programmer’s inequality symbol which is ”!=”. So yes, it is not easily readable (i! is a bad combination...) but totally correct.
A space between variable & operator would help.
The symbol wasn’t there when I wrote my comment. It was edited in afterwards.
The way to handle that is whitespace:
i != 0. (I once was teased by my tendency to put whitespace in computer code around all operators which would be spaced in typeset mathematical formulas.)EDIT: I also use italics for variables, boldface for vectors, etc. when handwriting. Whenever I get a new pen I immediately check whether it’s practical to do boldface with it.
Of course, an infitesimal prior dominating the posterior pdf might also be a hint that your model needs adjustment.