I disagree with Tyrrell (see below), but I can give a version of Tyrrell’s “trivial” formalization:
We want to show that:
Averaging over all theories T,
P(T makes correct predictions | T passes 10 tests) >
P(T makes correct predictions)
By Bayes’ rule,
P(T makes correct predictions | T passes 10 tests) =
P(T makes correct predictions)
P(T passes 10 tests | T makes correct predictions)
/ P(T passes 10 tests)
So our conclusion is equivalent to:
Averaging over all theories T,
P(T passes 10 tests | T makes correct predictions)
/ P(T passes 10 tests)
1
which is equivalent to
Averaging over all theories T,
P(T passes 10 tests | T makes correct predictions) > P(T passes 10 tests)
which has to be true for any plausible definition of “makes correct predictions”. The effect is only small if nearly all theories can pass the 10 tests.
I disagree with Tyrrell’s conclusion. I think his fallacy is to work with the undefined concept of “the best theory”, and to assume that:
If a theory consistent with past observations makes incorrect predictions then there was something wrong with the process by which that theory was formed. (Not true; making predictions is inherently an unreliable process.)
Therefore we can assume that that process produces bad theories with a fixed frequency. (Not meaningful; the observations made so far are a varying input to the process of forming theories.)
In the math above, the fallacy shows up because the set of theories that are consistent with the first 10 observations is different from the set of theories that are consistent with the first 20 observations, so the initial statement isn’t really what we wanted to show. (If that fallacy is a problem with my understanding of Tyrrell’s post, he should have done the “trivial” formalization himself.)
There are lots of ways to apply Bayes’ Rule, and this wasn’t the first one I tried, so I also disagree with Tyrrell’s claim that this is trivial.
Benja --
I disagree with Tyrrell (see below), but I can give a version of Tyrrell’s “trivial” formalization:
We want to show that:
Averaging over all theories T, P(T makes correct predictions | T passes 10 tests) > P(T makes correct predictions)
By Bayes’ rule,
P(T makes correct predictions | T passes 10 tests) = P(T makes correct predictions)
P(T passes 10 tests | T makes correct predictions) / P(T passes 10 tests)
So our conclusion is equivalent to:
Averaging over all theories T, P(T passes 10 tests | T makes correct predictions) / P(T passes 10 tests)
which is equivalent to
Averaging over all theories T, P(T passes 10 tests | T makes correct predictions) > P(T passes 10 tests)
which has to be true for any plausible definition of “makes correct predictions”. The effect is only small if nearly all theories can pass the 10 tests.
I disagree with Tyrrell’s conclusion. I think his fallacy is to work with the undefined concept of “the best theory”, and to assume that:
If a theory consistent with past observations makes incorrect predictions then there was something wrong with the process by which that theory was formed. (Not true; making predictions is inherently an unreliable process.)
Therefore we can assume that that process produces bad theories with a fixed frequency. (Not meaningful; the observations made so far are a varying input to the process of forming theories.)
In the math above, the fallacy shows up because the set of theories that are consistent with the first 10 observations is different from the set of theories that are consistent with the first 20 observations, so the initial statement isn’t really what we wanted to show. (If that fallacy is a problem with my understanding of Tyrrell’s post, he should have done the “trivial” formalization himself.)
There are lots of ways to apply Bayes’ Rule, and this wasn’t the first one I tried, so I also disagree with Tyrrell’s claim that this is trivial.