I think the intended meaning (phrased in LessWrong terminology) is something more along the lines of the following:
Humans are not perfect Bayesians, and even if they were, they don’t start from the same priors and encounter the same evidence. Therefore, Aumann’s Agreement Theorem does not hold for human beings; thus, if a large number of human beings is observed to agree on the truth of a proposition, you should be suspicious. It’s far more likely that they are signalling tribal agreement or, worse yet, accepting the proposition without thinking it through for themselves, than that they have each individually thought it through and independently reached identical conclusions. In general, then, civilized disagreement is a strong indicator of a healthy rationalist community; look at how often people disagree with each other on LW, for example. If everyone on LW was chanting, “Yes, Many Worlds is true, you should prefer torture to dust specks, mainstream philosophy is worthless,” then that would be worrying, even if it is true. (I am not claiming that it is, nor am I claiming that it is not; such topics are, I feel, beyond the scope of this discussion and were brought up purely as examples.)
I think the intended meaning (phrased in LessWrong terminology) is something more along the lines of the following:
Humans are not perfect Bayesians, and even if they were, they don’t start from the same priors and encounter the same evidence. Therefore, Aumann’s Agreement Theorem does not hold for human beings; thus, if a large number of human beings is observed to agree on the truth of a proposition, you should be suspicious. It’s far more likely that they are signalling tribal agreement or, worse yet, accepting the proposition without thinking it through for themselves, than that they have each individually thought it through and independently reached identical conclusions. In general, then, civilized disagreement is a strong indicator of a healthy rationalist community; look at how often people disagree with each other on LW, for example. If everyone on LW was chanting, “Yes, Many Worlds is true, you should prefer torture to dust specks, mainstream philosophy is worthless,” then that would be worrying, even if it is true. (I am not claiming that it is, nor am I claiming that it is not; such topics are, I feel, beyond the scope of this discussion and were brought up purely as examples.)