Hi everyone, my name is Jesse. I was introduced to LessWrong by my sister, Julia, a couple years ago and I’ve found the posts here fantastic.
Since college, I’ve been a professional atheist. I’ve done communications/PR work for three secular nonprofit organizations, helping to put a friendly face on nontheistic people and promoting a secular worldview/philosophy. It doesn’t exactly pay well, but I like knowing that I’m part of making the world a more rational place.
I’m fascinated by a lot of the same things you are—psychology, rationality, language. But as a communications director, I have a particular passion for effective communication and persuasion. The “A Human’s Guide to Words” sequence was invaluable in shaping my understanding and practice.
The question currently on my mind (among others) is:
“Does it make sense to call a particular persuasion technique unethical? Or does it entirely depend on how it’s used?”
Let me know what you think, and I look forward to being a part of this community!
What would they think if they knew exactly what I was doing?
If lots of people used this technique, would the world be better off or worse off? Is that already happening and am I just keeping pace? Am I being substantially less evil than average?
Is this the sort of Dark Art that corrupts anything it touches (like telling people to have faith) or is it more neutral toward the content conveyed (like using colorful illustrations or having a handsome presenter speak a talk)?
(I’ve recently joked that SIAI should change its motto from “Don’t be jerks” to “Be less evil than Google”.)
“Am I making people stronger, or weaker?” That’s a very important question, and sometimes hard to get right.
Consider a theist for whom the belief in god is a fundamental aspect of his life, whose faith makes him strong because it gives him something to protect. Breaking (or weakening) his belief in god before he built himself a line of retreat can do much more harm than good.
What should be done is first building the line of retreat, showing him that even without a god, his life does not become pointless, his ethics won’t crumble to dust, and the thing he wants to protect is still worth protecting. And then, but only then, showing to him that his belief in god is not only unnecessary, but also making him weaker.
Regarding the second one, “What would [people] think if they knew exactly what I was doing?”—I absolutely agree that it’s important as a pragmatic issue. If someone will get upset by a technique—justified or not—we need to factor that into the decision to use it.
But do you think their discomfort is a sign that the technique is unethical in any meaningful sense, or merely socially frowned upon? Society tends to form its conventions for a reason, but those reasons aren’t necessarily tied to a consistent conception of morality.
That said, I agree that if people get upset by a practice, it’s a good warning sign that the practice could be unethical and merits careful thought. …Which could be exactly what you meant by asking the question.
By the way, I’m looking forward to meeting you at Skepticon next month—I’ll be moderating a panel you’ll be on!
a pragmatic issue. If someone will get upset by a technique—justified or not—we need to factor that into the decision to use it...discomfort is a sign that the technique is unethical in any meaningful sense
If people get upset by a technique, that is a harm, but if their suffering that harm has good consequences, upsetting them was, all else equal, a good thing to do. So upsetting people is always related to ethics as more than just a sign.
discomfort is a sign that the technique is unethical in any meaningful sense, or merely socially frowned upon?
Unethical things are frowned upon to the extent people feel (at some level) frowning impacts that sort of action; regarding blame:
But determinists don’t just ignore the very important differences between brain tumors and poor taste in music. Some biological phenomena, like poor taste in music, are encoded in such a way that they are extremely vulnerable to what we can call social influences: praise, condemnation, introspection, and the like. Other biological phenomena, like brain tumors, are completely immune to such influences. This allows us to develop a more useful model of blame.
The consequentialist model of blame is very different from the deontological model. Because all actions are biologically determined, none are more or less metaphysically blameworthy than others, and none can mark anyone with the metaphysical status of “bad person” and make them “deserve” bad treatment. Consequentialists don’t on a primary level want anyone to be treated badly, full stop; thus is it written: “Saddam Hussein doesn’t deserve so much as a stubbed toe.” But if consequentialists don’t believe in punishment for its own sake, they do believe in punishment for the sake of, well, consequences. Hurting bank robbers may not be a good in and of itself, but it will prevent banks from being robbed in the future. And, one might infer, although alcoholics may not deserve condemnation, societal condemnation of alcoholics makes alcoholism a less attractive option.
So here, at last, is a rule for which diseases we offer sympathy, and which we offer condemnation: if giving condemnation instead of sympathy decreases the incidence of the disease enough to be worth the hurt feelings, condemn; otherwise, sympathize. Though the rule is based on philosophy that the majority of the human race would disavow, it leads to intuitively correct consequences. Yelling at a cancer patient, shouting “How dare you allow your cells to divide in an uncontrolled manner like this; is that the way your mother raised you??!” will probably make the patient feel pretty awful, but it’s not going to cure the cancer. Telling a lazy person “Get up and do some work, you worthless bum,” very well might cure the laziness. The cancer is a biological condition immune to social influences; the laziness is a biological condition susceptible to social influences, so we try to socially influence the laziness and not the cancer.
Society often has good reasons behind its moral classifications.
Didn’t know the story behind that one, so thank you Know Your Meme. That’s the face he made when Bill O’Reilly said “You can’t explain why the tide goes in.”
First I thought “Oh great, another believer in n gods for n=0”, but after looking through your site I realized that it is much more about rationality and a secular approach to life, not just telling people that faith is a bad thing.
As for the morality of a particular persuasion technique, “do unto others...” is still a golden rule, despite its inherent biases and religious connotations.
“Does it make sense to call a particular persuasion technique unethical? Or does it entirely depend on how it’s used?”
You may or may not be aware, but this has been discussed at some length around these parts; Dark Arts is an okay summary. (Edit: A particularly good post on the subject is NTLing.) If you’ve already read it and think the topic could stand more elaboration, though, I’m with you.
I’ve come across some of this material, but haven’t read it in a systematic way. I very occasionally refer to persuasion as ‘the dark arts’ - I think that phrase/connection came from LW originally.
Earlier this year I gave a talk on the psychology of persuasion, synthesizing some of the fascinating studies that have been done. Rather than present the most blatant techniques as manipulation, I framed them as known weaknesses in our minds that could be exploited if we weren’t wary and aware. Thus: defense against dark arts. Combining rationality and Harry Potter! Hey, that would be a great fanfiction! (Yes, I’m aware of Harry Potter and the Methods of Rationality and have done my best to spread it far and wide.)
Thanks for the support regarding my job: I’ve loved doing it and hope to do more for the secular movement!
Are you volunteering for the post of LessWrong’s DADA professor? The space is open if you want it, though Yvain has previously submitted an application. It should also be noted that a certain someone doesn’t seem interested in the job (probably a good thing, on balance).
I would say that any persuasion technique that requires plain lies is unethical. Lies are contagious and break trust, while trust is required for any constructive communication.
Now, it may be a lesser evil in some situations. But a lesser evil is still evil, and should be avoided every time it can be. So yes, to me, you can call a technique itself unethical. Some exceptional situations may force you to do something unethical, because the alternatives are much worse, but that can be said to anything (you can always construct an hypothetical situation in which a given ethical rule will have to be broken), so if we want to keep that “ethical” word, we can apply it to something like openly lying.
That’s one useful way to make a distinction! And, honestly, probably the one I lean toward. That’s probably the way I’d use the words, but even so I’m trying to figure out whether there’s a sensible and coherent way to call a persuasion technique unethical as a reflection on the technique, rather than solely the consequences.
I’ve thought about it another way—if a particular technique is far easier (and more likely) to be used in a way that reduces utility than it is to use in a positive way, society should be wary of it, and perhaps call it an unethical practice. I’m thinking of some alleged pick-up artist techniques that are based on lowering a woman’s self-esteem and sense of self-worth. (Disclaimer: this is second or third-hand information about PUA, so I could be misrepresenting it. Regardless of whether it’s practiced by PUA, the hypothetical holds.)
I’m trying to figure out whether there’s a sensible and coherent way to call a persuasion technique unethical as a reflection on the technique, rather than solely the consequences.
The first step might be to back up and see whether there’s a perfectly coherent way to distinguish among persuasion techniques, in case that becomes important.
Sure, there aresensible ways to distinguish among them. But if you had a good idea of what your subject’s matter was like, and a good idea of how you would want it to be, and you had sufficient power, you could talk softly to them, or torture them, or disassemble their atoms and reshape them into a nearly identical version that had a few changed opinions, or barbecue them and feed them to a child and teach the child the opinions you wanted them to have. All four ways begin with an interlocutor and end with a person made out of mostly the same atoms thinking largely what you set out to have the person you are talking to think. (Note: I do not claim that for every mind, persuasion would work.) While these methods are distinct, there is a continuum of possibilities along the influence-manipulation-reconstruction-recycling axis.
I don’t think there is a solid, sharp boundary marking a difference in kind between “influence” and Dark Art style “manipulation”.
On slavery, which everyone agrees is always wrong...right?
Hi everyone, my name is Jesse. I was introduced to LessWrong by my sister, Julia, a couple years ago and I’ve found the posts here fantastic.
Since college, I’ve been a professional atheist. I’ve done communications/PR work for three secular nonprofit organizations, helping to put a friendly face on nontheistic people and promoting a secular worldview/philosophy. It doesn’t exactly pay well, but I like knowing that I’m part of making the world a more rational place.
I’m fascinated by a lot of the same things you are—psychology, rationality, language. But as a communications director, I have a particular passion for effective communication and persuasion. The “A Human’s Guide to Words” sequence was invaluable in shaping my understanding and practice.
The question currently on my mind (among others) is: “Does it make sense to call a particular persuasion technique unethical? Or does it entirely depend on how it’s used?”
Let me know what you think, and I look forward to being a part of this community!
Jesse
Some questions to ask:
Am I making people stronger, or weaker?
What would they think if they knew exactly what I was doing?
If lots of people used this technique, would the world be better off or worse off? Is that already happening and am I just keeping pace? Am I being substantially less evil than average?
Is this the sort of Dark Art that corrupts anything it touches (like telling people to have faith) or is it more neutral toward the content conveyed (like using colorful illustrations or having a handsome presenter speak a talk)?
(I’ve recently joked that SIAI should change its motto from “Don’t be jerks” to “Be less evil than Google”.)
“Am I making people stronger, or weaker?” That’s a very important question, and sometimes hard to get right.
Consider a theist for whom the belief in god is a fundamental aspect of his life, whose faith makes him strong because it gives him something to protect. Breaking (or weakening) his belief in god before he built himself a line of retreat can do much more harm than good.
What should be done is first building the line of retreat, showing him that even without a god, his life does not become pointless, his ethics won’t crumble to dust, and the thing he wants to protect is still worth protecting. And then, but only then, showing to him that his belief in god is not only unnecessary, but also making him weaker.
Great questions!
Regarding the second one, “What would [people] think if they knew exactly what I was doing?”—I absolutely agree that it’s important as a pragmatic issue. If someone will get upset by a technique—justified or not—we need to factor that into the decision to use it.
But do you think their discomfort is a sign that the technique is unethical in any meaningful sense, or merely socially frowned upon? Society tends to form its conventions for a reason, but those reasons aren’t necessarily tied to a consistent conception of morality.
That said, I agree that if people get upset by a practice, it’s a good warning sign that the practice could be unethical and merits careful thought. …Which could be exactly what you meant by asking the question.
By the way, I’m looking forward to meeting you at Skepticon next month—I’ll be moderating a panel you’ll be on!
If people get upset by a technique, that is a harm, but if their suffering that harm has good consequences, upsetting them was, all else equal, a good thing to do. So upsetting people is always related to ethics as more than just a sign.
Unethical things are frowned upon to the extent people feel (at some level) frowning impacts that sort of action; regarding blame:
Society often has good reasons behind its moral classifications.
Use your gut.
I just checked out the Skepticon list of speakers. Laughter was induced by the picture of David Silverman.
Didn’t know the story behind that one, so thank you Know Your Meme. That’s the face he made when Bill O’Reilly said “You can’t explain why the tide goes in.”
First I thought “Oh great, another believer in n gods for n=0”, but after looking through your site I realized that it is much more about rationality and a secular approach to life, not just telling people that faith is a bad thing.
As for the morality of a particular persuasion technique, “do unto others...” is still a golden rule, despite its inherent biases and religious connotations.
Bienvenidos, Jesse!
You may or may not be aware, but this has been discussed at some length around these parts; Dark Arts is an okay summary. (Edit: A particularly good post on the subject is NTLing.) If you’ve already read it and think the topic could stand more elaboration, though, I’m with you.
Oh, and “professional atheist”? Totally awesome.
Thanks for the tip!
I’ve come across some of this material, but haven’t read it in a systematic way. I very occasionally refer to persuasion as ‘the dark arts’ - I think that phrase/connection came from LW originally.
Earlier this year I gave a talk on the psychology of persuasion, synthesizing some of the fascinating studies that have been done. Rather than present the most blatant techniques as manipulation, I framed them as known weaknesses in our minds that could be exploited if we weren’t wary and aware. Thus: defense against dark arts. Combining rationality and Harry Potter! Hey, that would be a great fanfiction! (Yes, I’m aware of Harry Potter and the Methods of Rationality and have done my best to spread it far and wide.)
Thanks for the support regarding my job: I’ve loved doing it and hope to do more for the secular movement!
I think the best approach is to read the sequence on a Human’s Guide to Words before subject specific material.
In particular at least the first nine (until Neural Categories) and also Categorizing Has Consequences Where to Draw the Boundary and Words as Mental Paintbrush Handles.
/clears throat suggestively
Are you volunteering for the post of LessWrong’s DADA professor? The space is open if you want it, though Yvain has previously submitted an application. It should also be noted that a certain someone doesn’t seem interested in the job (probably a good thing, on balance).
That depends—would I die horribly and mysteriously after a year?
No, of course not! Whatever gave you that idea?
(You might be found in a closet with three fifth-year Hansonians, though...)
I would say that any persuasion technique that requires plain lies is unethical. Lies are contagious and break trust, while trust is required for any constructive communication.
Now, it may be a lesser evil in some situations. But a lesser evil is still evil, and should be avoided every time it can be. So yes, to me, you can call a technique itself unethical. Some exceptional situations may force you to do something unethical, because the alternatives are much worse, but that can be said to anything (you can always construct an hypothetical situation in which a given ethical rule will have to be broken), so if we want to keep that “ethical” word, we can apply it to something like openly lying.
Particular persuasion techniques are called different things depending on if they are used ethically.
That’s one useful way to make a distinction! And, honestly, probably the one I lean toward. That’s probably the way I’d use the words, but even so I’m trying to figure out whether there’s a sensible and coherent way to call a persuasion technique unethical as a reflection on the technique, rather than solely the consequences.
I’ve thought about it another way—if a particular technique is far easier (and more likely) to be used in a way that reduces utility than it is to use in a positive way, society should be wary of it, and perhaps call it an unethical practice. I’m thinking of some alleged pick-up artist techniques that are based on lowering a woman’s self-esteem and sense of self-worth. (Disclaimer: this is second or third-hand information about PUA, so I could be misrepresenting it. Regardless of whether it’s practiced by PUA, the hypothetical holds.)
The first step might be to back up and see whether there’s a perfectly coherent way to distinguish among persuasion techniques, in case that becomes important.
Sure, there are sensible ways to distinguish among them. But if you had a good idea of what your subject’s matter was like, and a good idea of how you would want it to be, and you had sufficient power, you could talk softly to them, or torture them, or disassemble their atoms and reshape them into a nearly identical version that had a few changed opinions, or barbecue them and feed them to a child and teach the child the opinions you wanted them to have. All four ways begin with an interlocutor and end with a person made out of mostly the same atoms thinking largely what you set out to have the person you are talking to think. (Note: I do not claim that for every mind, persuasion would work.) While these methods are distinct, there is a continuum of possibilities along the influence-manipulation-reconstruction-recycling axis.
I don’t think there is a solid, sharp boundary marking a difference in kind between “influence” and Dark Art style “manipulation”.
On slavery, which everyone agrees is always wrong...right?