This is the first article in my Bah-Humbug Sequence a.k.a. “Everything I Don’t Like Is A Defect/Defect Equilibrium”. Epistemic status: strong opinion weakly held, somewhat exaggerated for dramatic effect; I’m posting this here so that the ensuing discussion might help me clarify my position. Anyway, the time has now come for me to explain my overbearing attitude of cynicism towards all aspects of life. Why now, of all times? I hope to make that clear by the end.
You are asking me to believe a certain claim. There is a simple and easy thing you can do to prove its trustworthiness, and yet you have not done that. I am therefore entitled to [Weak Adversarial Argument] disregard your claim as of no evidentiary value / [Strong Adversarial Argument] believe the negation of your claim purely out of spite.
What’s going on here? Are these valid arguments?
It may help to give some examples:
The Hearsay Objection—In a court of law, if a witness X tries to testify that some other person Y said Z, in trying to establish the truth of Z, the opposing side may object. This objection takes the form: “The opposition has brought in X to prove Z by way of the fact that Y said Z. But X is not the most reliable witness they could have called, because they could have summoned Y instead. If they were genuinely seeking the truth as to Z, they would have done so; and yet we see that they did not. Therefore I insist that X’s testimony be stricken from the record.”
The Cynical Cryptographer—My company’s HR department emails me a link to an employee satisfaction survey. The email is quick to say “Your responses are anonymous”, and yet I notice that the survey link contains a bunch of gobbledegook like ?id=2815ec7e931410a5fb358588ee70ad8b. I think to myself: If this actually is anonymous, and not a sham to see which employees have attitude problems and should be laid off first, the HR department could have set up a Chaumian blind signature protocol to provably guarantee that my response cannot be linked to my name. But they didn’t, and so I conclude that this survey is a sham, and I won’t fill it out.
So, again, are these valid arguments? From a Bayesian perspective, not really:
X saying that Y said Z is not literally zero evidence of Z. If there is any chance >0 that X and Y are honest, then I must update at least somewhat towards the truth of Z upon hearing X’s testimony.
I’m pretty sure they don’t teach cryptography in business school. An honest HR department and a dishonest one have approximately equal likelihood (i.e. ε) of knowing what a “Chaumian blind signature” is and actually implementing it. Therefore, by Bayes’ theorem, etc.
To steelman the Adversarial Argument, we should understand it not as an ordinary passive attempt to “rationally” form an accurate world-model, but rather as a sort of acausal negotiation tactic, akin to one-boxing on Newcomb’s Problem. By adopting it, we hope to “influence” the behavior of adversaries (i.e. people who want to convince us of something but don’t share our interests) towards providing stronger evidence, and away from trying to deceive us.
Or, to put it another way, the Adversarial Argument may not be valid in general, but by proclaiming it loudly and often, we can make it valid (at least in certain contexts) and thus make distinguishing truth and falsehood easier. Because the Hearsay Objection is enforced in court, lawyers who want to prove Z will either introduce direct witnesses or drop the claim altogether. And perhaps (we can dream!) if the Cynical Cryptographer argument catches on, honest HR departments will find themselves compelled to add Chaumian blind signatures to their surveys in order to get any responses, making the sham surveys easy to spot.
(Aside: Even under this formulation, we might accept the Weak Adversarial Argument but reject the Strong Adversarial Argument—by adopting a rule that I’ll believe the opposite of what an untrustworthy-seeming person says, I’m now setting myself up to be deceived into believing P by a clever adversary who asserts ¬P in a deliberately sleazy way—whereupon I’ll congratulate myself for seeing through the trick! Is there any way around this?)
Now, returning to the template above, the premise that “there is a simple and easy thing you can do to prove its trustworthiness” is doing a lot of work. Your adversary will always contend that the thing you want them to do (calling witness Y, adding Chaumian signatures, etc.) is too difficult and costly to reasonably expect of them. This may or may not be true, but someone who’s trying to deceive you will claim such regardless of its truth, hoping that they can “blend in” among the honest ones.
At that point, the situation reduces to a contest of wills over who gets to grab how much of the surplus value from our interaction. What is my trust worth to you? How much personal cost will you accept in order to gain it?
We on LessWrong—at least, those who wish to communicate the ideas we discuss here with people who don’t already agree—should be aware of this dynamic. There may have been a time in history when charismatic authority or essays full of big words were enough to win people over, but that is far from our present reality. In our time, propaganda and misinformation are well-honed arts. People are “accustomed to a haze of plausible-sounding arguments” and are rightly skeptical of all of them. Why should they trust the ideas on LessWrong, of all things? If we think gaining their trust is important and valuable, how much personal cost are we willing to accept to that end?
Or, backing up further: Why should you trust what you read here?
Adversarial epistemology
This is the first article in my Bah-Humbug Sequence a.k.a. “Everything I Don’t Like Is A Defect/Defect Equilibrium”. Epistemic status: strong opinion weakly held, somewhat exaggerated for dramatic effect; I’m posting this here so that the ensuing discussion might help me clarify my position. Anyway, the time has now come for me to explain my overbearing attitude of cynicism towards all aspects of life. Why now, of all times? I hope to make that clear by the end.
What’s going on here? Are these valid arguments?
It may help to give some examples:
The Hearsay Objection—In a court of law, if a witness X tries to testify that some other person Y said Z, in trying to establish the truth of Z, the opposing side may object. This objection takes the form: “The opposition has brought in X to prove Z by way of the fact that Y said Z. But X is not the most reliable witness they could have called, because they could have summoned Y instead. If they were genuinely seeking the truth as to Z, they would have done so; and yet we see that they did not. Therefore I insist that X’s testimony be stricken from the record.”
The Cynical Cryptographer—My company’s HR department emails me a link to an employee satisfaction survey. The email is quick to say “Your responses are anonymous”, and yet I notice that the survey link contains a bunch of gobbledegook like
?id=2815ec7e931410a5fb358588ee70ad8b. I think to myself: If this actually is anonymous, and not a sham to see which employees have attitude problems and should be laid off first, the HR department could have set up a Chaumian blind signature protocol to provably guarantee that my response cannot be linked to my name. But they didn’t, and so I conclude that this survey is a sham, and I won’t fill it out.So, again, are these valid arguments? From a Bayesian perspective, not really:
X saying that Y said Z is not literally zero evidence of Z. If there is any chance >0 that X and Y are honest, then I must update at least somewhat towards the truth of Z upon hearing X’s testimony.
I’m pretty sure they don’t teach cryptography in business school. An honest HR department and a dishonest one have approximately equal likelihood (i.e. ε) of knowing what a “Chaumian blind signature” is and actually implementing it. Therefore, by Bayes’ theorem, etc.
To steelman the Adversarial Argument, we should understand it not as an ordinary passive attempt to “rationally” form an accurate world-model, but rather as a sort of acausal negotiation tactic, akin to one-boxing on Newcomb’s Problem. By adopting it, we hope to “influence” the behavior of adversaries (i.e. people who want to convince us of something but don’t share our interests) towards providing stronger evidence, and away from trying to deceive us.
Or, to put it another way, the Adversarial Argument may not be valid in general, but by proclaiming it loudly and often, we can make it valid (at least in certain contexts) and thus make distinguishing truth and falsehood easier. Because the Hearsay Objection is enforced in court, lawyers who want to prove Z will either introduce direct witnesses or drop the claim altogether. And perhaps (we can dream!) if the Cynical Cryptographer argument catches on, honest HR departments will find themselves compelled to add Chaumian blind signatures to their surveys in order to get any responses, making the sham surveys easy to spot.
(Aside: Even under this formulation, we might accept the Weak Adversarial Argument but reject the Strong Adversarial Argument—by adopting a rule that I’ll believe the opposite of what an untrustworthy-seeming person says, I’m now setting myself up to be deceived into believing P by a clever adversary who asserts ¬P in a deliberately sleazy way—whereupon I’ll congratulate myself for seeing through the trick! Is there any way around this?)
Now, returning to the template above, the premise that “there is a simple and easy thing you can do to prove its trustworthiness” is doing a lot of work. Your adversary will always contend that the thing you want them to do (calling witness Y, adding Chaumian signatures, etc.) is too difficult and costly to reasonably expect of them. This may or may not be true, but someone who’s trying to deceive you will claim such regardless of its truth, hoping that they can “blend in” among the honest ones.
At that point, the situation reduces to a contest of wills over who gets to grab how much of the surplus value from our interaction. What is my trust worth to you? How much personal cost will you accept in order to gain it?
We on LessWrong—at least, those who wish to communicate the ideas we discuss here with people who don’t already agree—should be aware of this dynamic. There may have been a time in history when charismatic authority or essays full of big words were enough to win people over, but that is far from our present reality. In our time, propaganda and misinformation are well-honed arts. People are “accustomed to a haze of plausible-sounding arguments” and are rightly skeptical of all of them. Why should they trust the ideas on LessWrong, of all things? If we think gaining their trust is important and valuable, how much personal cost are we willing to accept to that end?
Or, backing up further: Why should you trust what you read here?