Can you be clearer? Log likelihood ratios do add up, so long as the independence criterion is satisfied (ie so long as P(E_2|H_x) = P(E_2|E_1,H_x) for each H_x).
Sure, just edited in the clarification: “you have to multiply odds by likelihood ratios, not odds by odds, and likewise you don’t add log odds and log odds, but log odds and log likelihood-ratios”.
It explains “mutual information”, i.e. “informational evidence”, which can be added up over as many independent events as you like. Hopefully this will have restorative effects for your intuition!
Can you be clearer? Log likelihood ratios do add up, so long as the independence criterion is satisfied (ie so long as P(E_2|H_x) = P(E_2|E_1,H_x) for each H_x).
Sure, just edited in the clarification: “you have to multiply odds by likelihood ratios, not odds by odds, and likewise you don’t add log odds and log odds, but log odds and log likelihood-ratios”.
As long as there are only two H_x, mind you. They no longer add up when you have three hypotheses or more.
Indeed—though I find it very hard to hang on to my intuitive grasp of this!
Here is the post on information theory I said I would write:
http://lesswrong.com/lw/1y9/information_theory_and_the_symmetry_of_updating/
It explains “mutual information”, i.e. “informational evidence”, which can be added up over as many independent events as you like. Hopefully this will have restorative effects for your intuition!
Don’t worry, I have an information theory post coming up that will fix all of this :)