we can run some simple statistical checks on the hypothesis and the data to see if our prior was wrong. For example, plot the data as a histogram, and plot the hypothesis as another histogram, and if there’s a lot of data and the two histograms are wildly different, we know almost for certain that the prior was wrong.
That check should be part of updating your prior. If you updated and got a hypothesis that didn’t fit the data, you didn’t update very well. You need to take this into account when you’re updating (and you also need to take into account the possibility of experimental error: there’s a small chance the data are wrong).
Hopefully the Book Club will get around to covering that as part of Chapter 4.
I can’t recall that it has anything to do with “updating your prior”; Jaynes just says that if you get nonsense posterior probabilities, you need to go back and include additional hypotheses in the set you’re considering, and this changes the analysis.
See also the quote (I can’t be bothered to find it now but I posted it a while ago to a quotes thread) where Jaynes says probability theory doesn’t do the job of thinking up hypotheses for you.
That check should be part of updating your prior. If you updated and got a hypothesis that didn’t fit the data, you didn’t update very well. You need to take this into account when you’re updating (and you also need to take into account the possibility of experimental error: there’s a small chance the data are wrong).
Hopefully the Book Club will get around to covering that as part of Chapter 4.
I can’t recall that it has anything to do with “updating your prior”; Jaynes just says that if you get nonsense posterior probabilities, you need to go back and include additional hypotheses in the set you’re considering, and this changes the analysis.
See also the quote (I can’t be bothered to find it now but I posted it a while ago to a quotes thread) where Jaynes says probability theory doesn’t do the job of thinking up hypotheses for you.