Unless you have actually used evidence to create a belief. If previous evidence actually supports a belief, unless you believe the real world is random, further evidence is more likely to support the belief than not.
That turns out not to be the case, though the reason why can initially seem unintuitive. If you have fully used all the information from piece of evidence A, that will include the fact that correlated piece of evidence B will be more likely to come up. This means that B will sway your beliefs less, because it is not a surprise. Contrariwise, anticorrelated piece of evidence C will be less likely to come up, and hence be more of a surprise if it does, and move your beliefs further. Averaging over all possible new pieces of evidence, and how likely they are, it has to be wash—if it’s not a wash, then you should have already updated to the point that would be your average expected update.
(Note that for something like parameter estimation, where rather than a single belief, you use probability densities, each point will on average stay the same for any new piece of evidence, but which parts of the density go up and which go down, and by how much are highly correlated.)
That turns out not to be the case, though the reason why can initially seem unintuitive. If you have fully used all the information from piece of evidence A, that will include the fact that correlated piece of evidence B will be more likely to come up. This means that B will sway your beliefs less, because it is not a surprise. Contrariwise, anticorrelated piece of evidence C will be less likely to come up, and hence be more of a surprise if it does, and move your beliefs further. Averaging over all possible new pieces of evidence, and how likely they are, it has to be wash—if it’s not a wash, then you should have already updated to the point that would be your average expected update.
(Note that for something like parameter estimation, where rather than a single belief, you use probability densities, each point will on average stay the same for any new piece of evidence, but which parts of the density go up and which go down, and by how much are highly correlated.)