I would essentially agree with this but refine that the real two competing theses are procedural about what we should do at point B in the argument:
Rule B1: If a significant group of people do a thing, then this in itself may be brought up as evidence that the thing may perhaps be rational, it is not necessary to further develop a thesis about how this group is reasoning correctly. A special thesis may be brought that on this occasion, people are acting irrationally, but this is burdensome and never quite believable with confidence except with the most extreme evidence.
Rule B2: If a significant group of people do a thing, this is an interesting observation, but there are many reasons why people do things, and to feel a slight sense of nervousness at departing their behavior pattern is leftover hunter-gatherer instinct which would poorly serve many of us now. To suppose that the group is acting rationally is a significant and unimplied further statement, which should not be made without specific supporting evidence especially if there seems to be a countervailing object-level argument.
(Hidden incentives which explain why people do what they do are commonplace, but unconscious reasoning will rarely add up to long-term rationality with respect to the original goal criterion being considered. It is both ‘cleverness’ and great implausibility to construct some elaborate pattern of secret knowledge which no one ever speaks explicitly, and Machiavellianness, and unusual personal goals or redefinitions of success, whereby the apparently stupid becomes smart.)
Object level arguments aside, people engaging in behavior A is still nontrivial evidence that behavior A is rational. Sure, it may be weak evidence, and can easily be swamped by object level arguments, but it can’t be entirely discarded.
The dichotomy Rule B1 vs. Rule B2 is a false dichotomy – one can be pretty confident that people are acting irrationally in a given instance even when one isn’t extremely confident, etc.
I would essentially agree with this but refine that the real two competing theses are procedural about what we should do at point B in the argument:
Rule B1: If a significant group of people do a thing, then this in itself may be brought up as evidence that the thing may perhaps be rational, it is not necessary to further develop a thesis about how this group is reasoning correctly. A special thesis may be brought that on this occasion, people are acting irrationally, but this is burdensome and never quite believable with confidence except with the most extreme evidence.
Rule B2: If a significant group of people do a thing, this is an interesting observation, but there are many reasons why people do things, and to feel a slight sense of nervousness at departing their behavior pattern is leftover hunter-gatherer instinct which would poorly serve many of us now. To suppose that the group is acting rationally is a significant and unimplied further statement, which should not be made without specific supporting evidence especially if there seems to be a countervailing object-level argument.
(Hidden incentives which explain why people do what they do are commonplace, but unconscious reasoning will rarely add up to long-term rationality with respect to the original goal criterion being considered. It is both ‘cleverness’ and great implausibility to construct some elaborate pattern of secret knowledge which no one ever speaks explicitly, and Machiavellianness, and unusual personal goals or redefinitions of success, whereby the apparently stupid becomes smart.)
Object level arguments aside, people engaging in behavior A is still nontrivial evidence that behavior A is rational. Sure, it may be weak evidence, and can easily be swamped by object level arguments, but it can’t be entirely discarded.
The dichotomy Rule B1 vs. Rule B2 is a false dichotomy – one can be pretty confident that people are acting irrationally in a given instance even when one isn’t extremely confident, etc.
Strong evidence or it didn’t happen.
That is my default reaction to this concept, having seen it so often on LW applied to unmeasurably small wisps of evidence.
Compare But There’s Still A Chance, Right?. “But It’s Still Evidence, Right?” is the other side of that dud coin.