I understood all of the other examples, but this one confused me:
A scenario is likely if it explains the data well. For example, many conspiracy theories are very likely because they have an answer for every question: a powerful group is conspiring to cover up the truth, meaning that the evidence we see is exactly what they’d want us to see.
If the conspiracy theory really was very likely, then we should be updating on this to have a higher posterior probability on the conspiracy theory. But in almost all cases we don’t actually believe the conspiracy theory is any more likely than we started out with. I think what’s actually going on is the thing Eliezer talked about in Technical Explanation where the conspiracy theory originally has the probability mass very spread out across different outcomes, but then as soon as it learns the actual outcome, it retroactively concentrates the probability mass on that outcome. So I want to say that the conspiracy theory is both unlikely (because it did not make an advance prediction) and improbable (very low prior combined with the unlikeliness). I’m curious if you agree with that or if I’ve misunderstood the example somehow.
Before the data comes in, the conspiracy theorist may not have a lot of predictions, or may have a lot of wrong predictions.
After the data comes in, though, the conspiracy theorist will have all sorts of stories about why the data fits perfectly with their theory.
My intention in what you quote was to consider the conspiracy theory in its fulness, after it’s been all fleshed out. This is usually the version of conspiracy theories I see.
That second version of the theory will be very likely, but have a very low prior probability. And when someone finds a conspiracy theory like that convincing, part of what’s going on may be that they confuse likelihood and probability. “It all makes sense! All the details fit!”
Whereas the original conspiracy theorist is making a very different kind of mistake.
I understood all of the other examples, but this one confused me:
If the conspiracy theory really was very likely, then we should be updating on this to have a higher posterior probability on the conspiracy theory. But in almost all cases we don’t actually believe the conspiracy theory is any more likely than we started out with. I think what’s actually going on is the thing Eliezer talked about in Technical Explanation where the conspiracy theory originally has the probability mass very spread out across different outcomes, but then as soon as it learns the actual outcome, it retroactively concentrates the probability mass on that outcome. So I want to say that the conspiracy theory is both unlikely (because it did not make an advance prediction) and improbable (very low prior combined with the unlikeliness). I’m curious if you agree with that or if I’ve misunderstood the example somehow.
Ah, yeah, I agree with your story.
Before the data comes in, the conspiracy theorist may not have a lot of predictions, or may have a lot of wrong predictions.
After the data comes in, though, the conspiracy theorist will have all sorts of stories about why the data fits perfectly with their theory.
My intention in what you quote was to consider the conspiracy theory in its fulness, after it’s been all fleshed out. This is usually the version of conspiracy theories I see.
That second version of the theory will be very likely, but have a very low prior probability. And when someone finds a conspiracy theory like that convincing, part of what’s going on may be that they confuse likelihood and probability. “It all makes sense! All the details fit!”
Whereas the original conspiracy theorist is making a very different kind of mistake.
Ah ok, that makes sense. Thanks for clarifying!