I noticed that I found it very difficult to read through this post, even though I felt the content was important, because of the (deliberately) condescending style. I also noticed that I’m finding it difficult to take the ideas as seriously as I think I should, again due to the style. I did manage to read through it in the end, because I do think it’s important, and I think I am mostly able to avoid letting the style influence my judgments. But I find it fascinating to watch my own reaction to the post, and I’m wondering if others have any (constructive) insights on this.
In general I I’ve noticed that I have a very hard time reading things that are written in a polemical, condescending, insulting, or ridiculing manner. This is particularly true of course if the target is a group / person / idea that I happen to like. But even if it’s written by someone on “my side” I find I have a hard time getting myself to read it. There have been several times when I’ve been told I should really go read a certain book, blog, article, etc., and that it has important content I should know about, but I couldn’t get myself to read the whole thing due to the polemical or insulting way in which it was written.
Similarly, as I noted above, I’ve noticed that I often have a hard time taking ideas as seriously as I probably should if they’re written in a polemical / condescending / insulting / ridiculing style. I think maybe I tend to down-weight the credibility of anybody who writes like that, and by extension maybe I subconsciously down-weight the content? Maybe I’m subconsciously associating condescension (at least towards ideas / people I think of as worth taking seriously) with bias? Not sure.
I’ve heard from other people that they especially like polemical / condescending articles, and I imagine that it is effective / persuasive for a lot of readers. For all I know this is far and away the most effective way of writing this kind of thing. And even if not, Eliezer is perfectly within his rights to use whatever style he wants. Eliezer explicitly acknowledges the condescending-sounding tone of the article, but felt it was worth writing it that way anyway, and that’s fine.
So to be clear: This is not at all a criticism of the way this post was written. I am simply curious about my own reaction to it, and I’m interested to hear what others think about that.
A few questions:
Am I unusual in this? Do other people here find it difficult to read polemical or condescending writing, and/or do you find that the style makes it difficult for you to take the content as seriously as you perhaps should?
Are there any studies you’re aware of on how people react to polemical writing?
Are there some situations in which it actually does make sense to use the kind of intuitive heuristic I was using—i.e., if it’s written in a polemical / insulting style then it’s probably less credible? Or is this just a generally bad heuristic that I should try to get rid of entirely?
This is a topic I’m very interested in so I’d appreciate any other related comments or thoughts you might have.
Things I instinctively observed slash that my model believes that I got while reading that seem relevant, not attempting to justify them at this time:
There is a core thing that Eliezer is trying to communicate. It’s not actually about timeline estimates, that’s an output of the thing. Its core message length is short, but all attempts to find short ways of expressing it, so far, have failed.
Mostly so have very long attempts to communicate it and its prerequisites, which to some extent at least includes the Sequences. Partial success in some cases, full success in almost none.
This post, and this whole series of posts, feels like its primary function is training data to use to produce an Inner Eliezer that has access to the core thing, or even better to know the core thing in a fully integrated way. And maybe a lot of Eliezer’s other communications is kind of also trying to be similar training data, no matter the superficial domain it is in or how deliberate that is.
The condescension is important information to help a reader figure out what is producing the outputs, and hiding it would make the task of ‘extract the key insights’ harder.
Similarly, the repetition of the same points is also potentially important information that points towards the core message.
That doesn’t mean all that isn’t super annoying to read and deal with, especially when he’s telling you in particular that you’re wrong. Cause it’s totally that.
There are those for whom this makes it easier to read, especially given it is very long, and I notice both effects.
My Inner Eliezer says that writing this post without the condescension, or making it shorter, would be much much more effort for Eliezer to write. To the extent such a thing can be written, someone else has to write that version. Also, it’s kind of text in several places.
The core message is what matters and the rest mostly doesn’t?
I am arrogant enough to think I have a non-zero chance that I know enough of the core thing and have enough skill that with enough work I could perhaps find an improved way to communicate it given the new training data, and I have the urge to try this impossible-level problem if I could find the time and focus (and help) to make a serious attempt.
I also stumbled on this point. I think it parses as
[attempt paraphrasing Zvi]
My Inner Eliezer says, “Writing this post without the condescension, or making it shorter, would be much much more effort for Eliezer to write. To the extent such a thing can be written, someone else has to write that version.” Also, besides my Inner Eliezer saying that, the preceding statement is almost explicit in the text in several places.
Evidence for the last bit is things like
[Eliezer’s OP]
Your grandpa is feeling kind of tired now and can’t debate this again with as much energy as when he was younger.
I endorse most of this comment; this “core thing” idea is exactly what I tried to understand when writing my recent post on deep knowledge according to Yudkowsky.
This post, and this whole series of posts, feels like its primary function is training data to use to produce an Inner Eliezer that has access to the core thing, or even better to know the core thing in a fully integrated way. And maybe a lot of Eliezer’s other communications is kind of also trying to be similar training data, no matter the superficial domain it is in or how deliberate that is.
Yeah, that sounds right. I feel like Yudkowsky always write mostly training data, and feels like explaining as precisely as he can the thing he’s talking about never works. I agree with him that it can’t work without the reader doing a bunch of work (what he calls homework), but I expect (from my personal experience) that doing the work while you have an outline of the thing is significantly easier. It’s easier to trust that there’s something valuable at the end of the tunnel when you have a half-decent description.
The condescension is important information to help a reader figure out what is producing the outputs, and hiding it would make the task of ‘extract the key insights’ harder.
Here though I feel like you’re overinterpreting. In older writing, Yudkowsky is actually quite careful to not directly insult people and be condescending. I’m not saying he never does it, but he tones it down a lot compared to what’s happening in this recent dialogue. I think that a better explanation is simply that he’s desperate, and has very little hope of being able to convey what he means because he’s being doing that for 13 years and no one catched on.
Maybe point 8 is also part of the explanation: doing this non-condescendingly sounds like far more work for him, and yet he does’t expect it to work, so he doesn’t take that extra charge for little expected reward.
My Inner Eliezer says that writing this post without the condescension, or making it shorter, would be much much more effort for Eliezer to write. To the extent such a thing can be written, someone else has to write that version. Also, it’s kind of text in several places.
I find it concerning that you felt the need to write “This is not at all a criticism of the way this post was written. I am simply curious about my own reaction to it” (and still got downvoted?).
For my part, I both believe that this post contains valuable content and good arguments, and that it was annoying / rude / bothersome in certain sections.
I had a pretty strong negative reaction to it. I got the feeling that the post derives much of its rhetorical force from setting up an intentionally stupid character who can be condescended to, and that this is used to sneak in a conclusion that would seem much weaker without that device.
When I try to mentally simulate negative reader-reactions to the dialogue, I usually get a complicated feeling that’s some combination of:
Some amount of conflict aversion: Harsh language feels conflict-y, which is inherently unpleasant.
Empathy for, or identification with, the people or views Eliezer was criticizing. It feels bad to be criticized, and it feels doubly bad to be told ‘you are making basic mistakes’.
Something status-regulation-y: My reader-model here finds the implied threat to the status hierarchy salient (whether or not Eliezer is just trying to honestly state his beliefs), and has some version of an ‘anti-cheater’ or ‘anti-rising-above-your-station’ impulse.
How right/wrong do you think this is, as a model of what makes the dialogue harder or less pleasant to read from your perspective?
(I feel a little wary of stating my model above, since (a) maybe it’s totally off, and (b) it can be rude to guess at other people’s mental states. But so far this conversation has felt very abstract to me, so maybe this can at least serve as a prompt to go more concrete. E.g., ‘I find it hard to read condescending things’ is very vague about which parts of the dialogue we’re talking about, about what makes them feel condescending, and about how the feeling-of-condescension affects the sentence-parsing-and-evaluating experience.)
I think part of what I was reacting to is a kind of half-formed argument that goes something like:
My prior credence is very low that all these really smart, carefully thought-through people are making the kinds of stupid or biased mistakes they are being accused of.
In fact, my prior for the above is sufficiently low that I suspect it’s more likely that the author is the one making the mistake(s) here, at least in the sense of straw-manning his opponents.
But if that’s the case then I shouldn’t trust the other things he says as much, because it looks like he’s making reasoning mistakes himself or else he’s biased.
Therefore I shouldn’t take his arguments so seriously.
Again, this isn’t actually an argument I would make. It’s just me trying to articulate my initial negative reactions to the post.
Right. And according to Zvi’s posit above, a large part of the point of this dialog is that that class of implicit argument is not actually good reasoning (acknowledging that you don’t endorse this argument).
More specifically, says my Inner Eliezer, it is less helpful to reason from or about one’s priors about really smart, careful-thinking people making or not making mistakes, and much more helpful to think directly about the object-level arguments, and whether they seem true.
“More specifically, says my Inner Eliezer, it is less helpful to reason from or about one’s priors about really smart, careful-thinking people making or not making mistakes, and much more helpful to think directly about the object-level arguments, and whether they seem true.”
When you say it’s much more helpful, do you mean it’s helpful for (a) forming accurate credences about which side is in fact correct, or do you just mean it’s helpful for (b) getting a much deeper understanding of the issues? If (b) then I totally agree. If (a) though, why would I expect myself to achieve a more accurate credence about the true state of affairs than any of the people in this argument? If it’s because they’ve stated their arguments for all the world to see so now anybody can go assess those arguments—why should I think I can better assess those arguments than Eliezer and his interlocutors? They clearly still disagree with each other despite reading all the same things I’m reading (and much more, actually). And add to that the fact that Eliezer is essentially saying in these dialogues that he has private reasoning and arguments that he cannot properly express and nobody seems to understand, in which case we have no choice but to do a secondary assessment of how likely he is to have good arguments of that type, or else to form our credences while completely ignoring the possible existence of a very critical argument in one direction.
Sometimes assessments of the argument maker’s cognitive abilities and access to relevant knowledge / expertise is in fact the best way to get the most accurate credence you can, even if it’s not ideal.
(This is all just repeating standard arguments in favor of modest epistemology, but still.)
I had mixed feelings about the dialogue personally. I enjoy the writing style and think Eliezer is a great writer with a lot of good opinions and arguments, which made it enjoyable.
But at the same time, it felt like he was taking down a strawman. Maybe you’d label it part of “conflict aversion”, but I tend to get a negative reaction to take-downs of straw-people who agree with me.
To give an unfair and exaggerated comparison, it would be a bit like reading a take-down of a straw-rationalist in which the straw-rationalist occasionally insists such things as “we should not be emotional” or “we should always use Bayes’ Theorem in every problem we encounter.” It should hopefully be easy to see why a rationalist might react negatively to reading that sort of dialogue.
1: To me, it made it more entertaining and thus easier to read. (No idea about non-anecdotal data, would also be interested.)
3: Also no data; I strongly suspect the metric is generally good because… actually I think it’s just because the people I find worth listening to are overwhelmingly not condescending. This post seems highly usual in several ways.
My posting this comment will be contrary to the moderation disclaimer advising not to talk about tone. But FWIW, I react similarly and I skip reading things written in this way, interpreting them as manipulating me into believing the writer is hypercompetent.
It’s not just a meta issue. The way it’s written has a big impact on how to engage with it.
In general I I’ve noticed that I have a very hard time reading things that are written in a polemical, condescending, insulting, or ridiculing manner. This is particularly true of course if the target is a group / person / idea that I happen to like. But even if it’s written by someone on “my side” I find I have a hard time getting myself to read it. There have been several times when I’ve been told I should really go read a certain book, blog, article, etc., and that it has important content I should know about, but I couldn’t get myself to read the whole thing due to the polemical or insulting way in which it was written.
I dealt with this by reading it and trying to be critical. The comment this produced was (predictably) downvoted.
The size of the community working on the alignment problem can be assumed to be at least somewhat proportional to the likelihood of successfully solving the alignment problem.
Eliezer, being the most public face of the alignment problem community, wields outsized influence in shaping public perception of the community.
Eliezer’s writing is distinctly condescending and polemical, and has at least a hypothetical possibility of causing reputational harm to the community (as evidenced by your comment).
Based on this, there absolutely exists a hypothetical point where, based purely on writing style, the net effect of a post like this could fully undermine the post’s ostensible aim. Whether this post crosses that point is a subjective evaluation, and I don’t know of any rigorous way to evaluate this.
I’m fully aware that this could be construed as “tone policing”, but ignorance of the impacts of writing tone seems like a blind spot to Eliezer and the community overall, so I think the topic is worthy of discussion.
Meta-comment:
I noticed that I found it very difficult to read through this post, even though I felt the content was important, because of the (deliberately) condescending style. I also noticed that I’m finding it difficult to take the ideas as seriously as I think I should, again due to the style. I did manage to read through it in the end, because I do think it’s important, and I think I am mostly able to avoid letting the style influence my judgments. But I find it fascinating to watch my own reaction to the post, and I’m wondering if others have any (constructive) insights on this.
In general I I’ve noticed that I have a very hard time reading things that are written in a polemical, condescending, insulting, or ridiculing manner. This is particularly true of course if the target is a group / person / idea that I happen to like. But even if it’s written by someone on “my side” I find I have a hard time getting myself to read it. There have been several times when I’ve been told I should really go read a certain book, blog, article, etc., and that it has important content I should know about, but I couldn’t get myself to read the whole thing due to the polemical or insulting way in which it was written.
Similarly, as I noted above, I’ve noticed that I often have a hard time taking ideas as seriously as I probably should if they’re written in a polemical / condescending / insulting / ridiculing style. I think maybe I tend to down-weight the credibility of anybody who writes like that, and by extension maybe I subconsciously down-weight the content? Maybe I’m subconsciously associating condescension (at least towards ideas / people I think of as worth taking seriously) with bias? Not sure.
I’ve heard from other people that they especially like polemical / condescending articles, and I imagine that it is effective / persuasive for a lot of readers. For all I know this is far and away the most effective way of writing this kind of thing. And even if not, Eliezer is perfectly within his rights to use whatever style he wants. Eliezer explicitly acknowledges the condescending-sounding tone of the article, but felt it was worth writing it that way anyway, and that’s fine.
So to be clear: This is not at all a criticism of the way this post was written. I am simply curious about my own reaction to it, and I’m interested to hear what others think about that.
A few questions:
Am I unusual in this? Do other people here find it difficult to read polemical or condescending writing, and/or do you find that the style makes it difficult for you to take the content as seriously as you perhaps should?
Are there any studies you’re aware of on how people react to polemical writing?
Are there some situations in which it actually does make sense to use the kind of intuitive heuristic I was using—i.e., if it’s written in a polemical / insulting style then it’s probably less credible? Or is this just a generally bad heuristic that I should try to get rid of entirely?
This is a topic I’m very interested in so I’d appreciate any other related comments or thoughts you might have.
Things I instinctively observed slash that my model believes that I got while reading that seem relevant, not attempting to justify them at this time:
There is a core thing that Eliezer is trying to communicate. It’s not actually about timeline estimates, that’s an output of the thing. Its core message length is short, but all attempts to find short ways of expressing it, so far, have failed.
Mostly so have very long attempts to communicate it and its prerequisites, which to some extent at least includes the Sequences. Partial success in some cases, full success in almost none.
This post, and this whole series of posts, feels like its primary function is training data to use to produce an Inner Eliezer that has access to the core thing, or even better to know the core thing in a fully integrated way. And maybe a lot of Eliezer’s other communications is kind of also trying to be similar training data, no matter the superficial domain it is in or how deliberate that is.
The condescension is important information to help a reader figure out what is producing the outputs, and hiding it would make the task of ‘extract the key insights’ harder.
Similarly, the repetition of the same points is also potentially important information that points towards the core message.
That doesn’t mean all that isn’t super annoying to read and deal with, especially when he’s telling you in particular that you’re wrong. Cause it’s totally that.
There are those for whom this makes it easier to read, especially given it is very long, and I notice both effects.
My Inner Eliezer says that writing this post without the condescension, or making it shorter, would be much much more effort for Eliezer to write. To the extent such a thing can be written, someone else has to write that version. Also, it’s kind of text in several places.
The core message is what matters and the rest mostly doesn’t?
I am arrogant enough to think I have a non-zero chance that I know enough of the core thing and have enough skill that with enough work I could perhaps find an improved way to communicate it given the new training data, and I have the urge to try this impossible-level problem if I could find the time and focus (and help) to make a serious attempt.
I would very much like to read your attempt at conveying the core thing—if nothing else, it’ll give another angle from which to try to grasp it.
What did you mean by this?
I also stumbled on this point. I think it parses as
[attempt paraphrasing Zvi]
Evidence for the last bit is things like
[Eliezer’s OP]
etc.
I endorse most of this comment; this “core thing” idea is exactly what I tried to understand when writing my recent post on deep knowledge according to Yudkowsky.
Yeah, that sounds right. I feel like Yudkowsky always write mostly training data, and feels like explaining as precisely as he can the thing he’s talking about never works. I agree with him that it can’t work without the reader doing a bunch of work (what he calls homework), but I expect (from my personal experience) that doing the work while you have an outline of the thing is significantly easier. It’s easier to trust that there’s something valuable at the end of the tunnel when you have a half-decent description.
Here though I feel like you’re overinterpreting. In older writing, Yudkowsky is actually quite careful to not directly insult people and be condescending. I’m not saying he never does it, but he tones it down a lot compared to what’s happening in this recent dialogue. I think that a better explanation is simply that he’s desperate, and has very little hope of being able to convey what he means because he’s being doing that for 13 years and no one catched on.
Maybe point 8 is also part of the explanation: doing this non-condescendingly sounds like far more work for him, and yet he does’t expect it to work, so he doesn’t take that extra charge for little expected reward.
I find it concerning that you felt the need to write “This is not at all a criticism of the way this post was written. I am simply curious about my own reaction to it” (and still got downvoted?).
For my part, I both believe that this post contains valuable content and good arguments, and that it was annoying / rude / bothersome in certain sections.
I had a pretty strong negative reaction to it. I got the feeling that the post derives much of its rhetorical force from setting up an intentionally stupid character who can be condescended to, and that this is used to sneak in a conclusion that would seem much weaker without that device.
When I try to mentally simulate negative reader-reactions to the dialogue, I usually get a complicated feeling that’s some combination of:
Some amount of conflict aversion: Harsh language feels conflict-y, which is inherently unpleasant.
Empathy for, or identification with, the people or views Eliezer was criticizing. It feels bad to be criticized, and it feels doubly bad to be told ‘you are making basic mistakes’.
Something status-regulation-y: My reader-model here finds the implied threat to the status hierarchy salient (whether or not Eliezer is just trying to honestly state his beliefs), and has some version of an ‘anti-cheater’ or ‘anti-rising-above-your-station’ impulse.
How right/wrong do you think this is, as a model of what makes the dialogue harder or less pleasant to read from your perspective?
(I feel a little wary of stating my model above, since (a) maybe it’s totally off, and (b) it can be rude to guess at other people’s mental states. But so far this conversation has felt very abstract to me, so maybe this can at least serve as a prompt to go more concrete. E.g., ‘I find it hard to read condescending things’ is very vague about which parts of the dialogue we’re talking about, about what makes them feel condescending, and about how the feeling-of-condescension affects the sentence-parsing-and-evaluating experience.)
I think part of what I was reacting to is a kind of half-formed argument that goes something like:
My prior credence is very low that all these really smart, carefully thought-through people are making the kinds of stupid or biased mistakes they are being accused of.
In fact, my prior for the above is sufficiently low that I suspect it’s more likely that the author is the one making the mistake(s) here, at least in the sense of straw-manning his opponents.
But if that’s the case then I shouldn’t trust the other things he says as much, because it looks like he’s making reasoning mistakes himself or else he’s biased.
Therefore I shouldn’t take his arguments so seriously.
Again, this isn’t actually an argument I would make. It’s just me trying to articulate my initial negative reactions to the post.
Right. And according to Zvi’s posit above, a large part of the point of this dialog is that that class of implicit argument is not actually good reasoning (acknowledging that you don’t endorse this argument).
More specifically, says my Inner Eliezer, it is less helpful to reason from or about one’s priors about really smart, careful-thinking people making or not making mistakes, and much more helpful to think directly about the object-level arguments, and whether they seem true.
“More specifically, says my Inner Eliezer, it is less helpful to reason from or about one’s priors about really smart, careful-thinking people making or not making mistakes, and much more helpful to think directly about the object-level arguments, and whether they seem true.”
When you say it’s much more helpful, do you mean it’s helpful for (a) forming accurate credences about which side is in fact correct, or do you just mean it’s helpful for (b) getting a much deeper understanding of the issues? If (b) then I totally agree. If (a) though, why would I expect myself to achieve a more accurate credence about the true state of affairs than any of the people in this argument? If it’s because they’ve stated their arguments for all the world to see so now anybody can go assess those arguments—why should I think I can better assess those arguments than Eliezer and his interlocutors? They clearly still disagree with each other despite reading all the same things I’m reading (and much more, actually). And add to that the fact that Eliezer is essentially saying in these dialogues that he has private reasoning and arguments that he cannot properly express and nobody seems to understand, in which case we have no choice but to do a secondary assessment of how likely he is to have good arguments of that type, or else to form our credences while completely ignoring the possible existence of a very critical argument in one direction.
Sometimes assessments of the argument maker’s cognitive abilities and access to relevant knowledge / expertise is in fact the best way to get the most accurate credence you can, even if it’s not ideal.
(This is all just repeating standard arguments in favor of modest epistemology, but still.)
I had mixed feelings about the dialogue personally. I enjoy the writing style and think Eliezer is a great writer with a lot of good opinions and arguments, which made it enjoyable.
But at the same time, it felt like he was taking down a strawman. Maybe you’d label it part of “conflict aversion”, but I tend to get a negative reaction to take-downs of straw-people who agree with me.
To give an unfair and exaggerated comparison, it would be a bit like reading a take-down of a straw-rationalist in which the straw-rationalist occasionally insists such things as “we should not be emotional” or “we should always use Bayes’ Theorem in every problem we encounter.” It should hopefully be easy to see why a rationalist might react negatively to reading that sort of dialogue.
I’ve gotten one private message expressing more or less the same thing about this post, so I don’t think this is a super unusual reaction.
1: To me, it made it more entertaining and thus easier to read. (No idea about non-anecdotal data, would also be interested.)
3: Also no data; I strongly suspect the metric is generally good because… actually I think it’s just because the people I find worth listening to are overwhelmingly not condescending. This post seems highly usual in several ways.
My posting this comment will be contrary to the moderation disclaimer advising not to talk about tone. But FWIW, I react similarly and I skip reading things written in this way, interpreting them as manipulating me into believing the writer is hypercompetent.
It’s not just a meta issue. The way it’s written has a big impact on how to engage with it.
I dealt with this by reading it and trying to be critical. The comment this produced was (predictably) downvoted.
The size of the community working on the alignment problem can be assumed to be at least somewhat proportional to the likelihood of successfully solving the alignment problem.
Eliezer, being the most public face of the alignment problem community, wields outsized influence in shaping public perception of the community.
Eliezer’s writing is distinctly condescending and polemical, and has at least a hypothetical possibility of causing reputational harm to the community (as evidenced by your comment).
Based on this, there absolutely exists a hypothetical point where, based purely on writing style, the net effect of a post like this could fully undermine the post’s ostensible aim. Whether this post crosses that point is a subjective evaluation, and I don’t know of any rigorous way to evaluate this.
I’m fully aware that this could be construed as “tone policing”, but ignorance of the impacts of writing tone seems like a blind spot to Eliezer and the community overall, so I think the topic is worthy of discussion.
My first order response to this is in https://www.lesswrong.com/posts/Js34Ez9nrDeJCTYQL/politics-is-way-too-meta