if we try to do it the other way around and apply the theory to update the probabilities we got from the experiments, we would be able to convince ourselves the probability is 75% no matter how much empirical evidence that says otherwise we have accumulated.
If this were true, I would agree with you. I am very much on board with the idea that we are flawed and that we should take steps to minimize the impact of these flaws, even if those steps wouldn’t be necessary for a perfect Bayesian.
However, it isn’t at all apparent to me that your assumption is true. My intuition is that it wouldn’t make much of a difference. But this sounds like a great idea for a psychology/behavioral economics experiment!
The difference can be quite large. If we get the results first, we can come up with Fake Explanations why the masks were only 20% effective in the experiments where in reality they are 75% effective. If we do the prediction first, we wouldn’t predict 20% effectiveness. We wouldn’t predict that our experiment will “fail”. Our theory says masks are effective so we would predict 75% to begin with, and when we get the results it’ll put a big dent in our theory. As it should.
If this were true, I would agree with you. I am very much on board with the idea that we are flawed and that we should take steps to minimize the impact of these flaws, even if those steps wouldn’t be necessary for a perfect Bayesian.
However, it isn’t at all apparent to me that your assumption is true. My intuition is that it wouldn’t make much of a difference. But this sounds like a great idea for a psychology/behavioral economics experiment!
The difference can be quite large. If we get the results first, we can come up with Fake Explanations why the masks were only 20% effective in the experiments where in reality they are 75% effective. If we do the prediction first, we wouldn’t predict 20% effectiveness. We wouldn’t predict that our experiment will “fail”. Our theory says masks are effective so we would predict 75% to begin with, and when we get the results it’ll put a big dent in our theory. As it should.