A non-conscious intent? What on earth does this mean?
This sort of idea seems to me like “let’s psychoanalyze people we don’t like to paint them in a bad light—and in such a way that they’re incapable of defending themselves”. It’s thoroughly toxic, and corrosive to productive or even remotely sane interaction.
If you have the sense that a person is just trying to dominate you, you may be intimidated, but you don’t necessarily think “Oh this person is the kindest and most misunderstood person I’ve ever met, I better do everything possible to help them, OMG they are so good.” Or, alternatively, you don’t think “OMG this person seems so smart, perhaps misunderstood in some way, but they’ve got everything figured out and I can learn so much from them.”
My view is that if you ever find yourself thinking any such thing, immediately stop and remind yourself not to put people on pedestals. Don’t do everything possible to help someone you just met, and don’t conclude that someone’s “got everything figured out” and that you can “learn so much from them”. Just, in general—don’t. This is orthogonal to anyone’s intent or what have you; it’s just a bad idea to approach interpersonal interaction in this way, and it will predictably lead you into bad situations, and generally cause you problems.
There’s a literature on self-deception and hypocrisy in humans that I’m sure you’re aware of. By “non-conscious intent,” which I admit is a confusing/poor phrasing, I wanted to point to that cluster of things. I used the word “intent” because I meant to say that there’s some kind of optimization at work here (but again, “intent” is poor phrasing). Frame control, the way I think of it, looks like agentic behavior with a goal of gaining influence over the person in question. The optimization at work could be something evolution installed or something that people simply learned has desired effects, without necessarily understanding why it has those effects. (E.g., if “playing the victim” gives you lots of sympathy and attention, you may start to do this more often whether or not you’re explicitly aware that the situation isn’t black and white in terms of you being the victim.)
For instance, someone may feel extreme shame whenever they’re criticized, so their first instinct when criticized is to get outraged at the person who dares to bring something up, calling them out and trying to shame them, etc. Against certain susceptible people, that strategy works in that it makes them feel bad for bringing up criticism. But the person being criticized may not be thinking through these implications explicitly, instead, they’re (here) primarily experiencing shame and turn that into reactive rage. (This is more of the vulnerable narcissism phenotype. In other instances, the frame distortion happens more because someone seems immune to criticism because their regard of the other person is so low that they believe the only reason they’re being criticized is because others are jelous and/or too flawed to see their greatness.)
My view is that if you ever find yourself thinking any such thing, immediately stop and remind yourself not to put people on pedestals. Don’t do everything possible to help someone you just met, and don’t conclude that someone’s “got everything figured out” and that you can “learn so much from them”. Just, in general—don’t. This is orthogonal to anyone’s intent or what have you; it’s just a bad idea to approach interpersonal interaction in this way, and it will predictably lead you into bad situations, and generally cause you problems.
I agree this is good advice. But to some degree, and for people who are very agreeable by disposition, it can feel really nice to try to help someone a lot or think very highly of them. Still, you’re right: I think the people susceptible to frame distortion tend to have low self-esteem (or something related, I think “self-esteem” is a complicated concept and maybe not unidimensional) and are too prone to seeing themselves as in the wrong. (Or, in three-party interactions, the people who most easily buy into someone’s frame control, thinking that the abuser is actually the victim, are people who are highly empathetic, but insufficiently cynical.)
A non-conscious intent? What on earth does this mean?
A brain can run computations optimizing for an outcome without running the additional computations needed to represent this optimization target to itself in explicit self-models available to reflective cognition.
Robin Hanson’s developed extensive, detailed sociological models that include this component. I think that the entire Overcoming Bias archives, not just Eliezer’s Sequences, ought to be canonical here, both because of their intellectual merit and because most of the Sequences were originally written on the Overcoming Bias blog in dialogue with Robin and his other co-bloggers there. A Theory of Identity seems particularly relevant here, and develops a directly relevant claim:
Our conscious minds are the public relations department of our larger minds, presenting and managing a story of ourselves to others.
Sorry, what?
A non-conscious intent? What on earth does this mean?
This sort of idea seems to me like “let’s psychoanalyze people we don’t like to paint them in a bad light—and in such a way that they’re incapable of defending themselves”. It’s thoroughly toxic, and corrosive to productive or even remotely sane interaction.
My view is that if you ever find yourself thinking any such thing, immediately stop and remind yourself not to put people on pedestals. Don’t do everything possible to help someone you just met, and don’t conclude that someone’s “got everything figured out” and that you can “learn so much from them”. Just, in general—don’t. This is orthogonal to anyone’s intent or what have you; it’s just a bad idea to approach interpersonal interaction in this way, and it will predictably lead you into bad situations, and generally cause you problems.
There’s a literature on self-deception and hypocrisy in humans that I’m sure you’re aware of. By “non-conscious intent,” which I admit is a confusing/poor phrasing, I wanted to point to that cluster of things.
I used the word “intent” because I meant to say that there’s some kind of optimization at work here (but again, “intent” is poor phrasing). Frame control, the way I think of it, looks like agentic behavior with a goal of gaining influence over the person in question. The optimization at work could be something evolution installed or something that people simply learned has desired effects, without necessarily understanding why it has those effects. (E.g., if “playing the victim” gives you lots of sympathy and attention, you may start to do this more often whether or not you’re explicitly aware that the situation isn’t black and white in terms of you being the victim.)
For instance, someone may feel extreme shame whenever they’re criticized, so their first instinct when criticized is to get outraged at the person who dares to bring something up, calling them out and trying to shame them, etc. Against certain susceptible people, that strategy works in that it makes them feel bad for bringing up criticism. But the person being criticized may not be thinking through these implications explicitly, instead, they’re (here) primarily experiencing shame and turn that into reactive rage. (This is more of the vulnerable narcissism phenotype. In other instances, the frame distortion happens more because someone seems immune to criticism because their regard of the other person is so low that they believe the only reason they’re being criticized is because others are jelous and/or too flawed to see their greatness.)
I agree this is good advice. But to some degree, and for people who are very agreeable by disposition, it can feel really nice to try to help someone a lot or think very highly of them. Still, you’re right: I think the people susceptible to frame distortion tend to have low self-esteem (or something related, I think “self-esteem” is a complicated concept and maybe not unidimensional) and are too prone to seeing themselves as in the wrong. (Or, in three-party interactions, the people who most easily buy into someone’s frame control, thinking that the abuser is actually the victim, are people who are highly empathetic, but insufficiently cynical.)
A brain can run computations optimizing for an outcome without running the additional computations needed to represent this optimization target to itself in explicit self-models available to reflective cognition.
Robin Hanson’s developed extensive, detailed sociological models that include this component. I think that the entire Overcoming Bias archives, not just Eliezer’s Sequences, ought to be canonical here, both because of their intellectual merit and because most of the Sequences were originally written on the Overcoming Bias blog in dialogue with Robin and his other co-bloggers there. A Theory of Identity seems particularly relevant here, and develops a directly relevant claim: