It looks like this post is resting on defining “rationalist” as “one who studies the laws of rationality”, as opposed “someone training to think rationally”, but, hasn’t really acknowledged that it’s using this definition (when I think Duncan and Robby’s posts seem pointed more at the latter definition)
(Actually, looking more, I think this post sort of equivocates between the two, without noting that it’s done so).
I’m not 100% sure I’ve parsed this right, but, this looks at first glance like the sort of language trick that you (Zack) are often (rightfully) annoyed at.
(I think it’s a reasonable conversational move to point out someone else’s definition of a word isn’t the only valid definition, and pointing out their frame isn’t the only valid frame. But if you’re doing that it’s better to do that explicitly)
I’ll agree that the “physicist motors” analogy in particular rests on the “one who studies” definition, although I think a lot of the points I make in this essay don’t particularly depend on the analogy and could easily be written up separately.
I guess you could view the “foreign policy” motivating this post as being driven by two motives: first, I’d rather not waste precious time (in the year 2023, when a lot of us have more important things to do) fighting over the “rationalist” brand name; if someone else who also cares about thinking well, thinks that I’m going about everything all wrong, I think it’s fine that we just have our own separate dojos, Archipelago-style. That’s why the post emphasizes that there are many types of motors and many types of marital arts.
But secondly, insofar as I’m unfortunately stuck fighting over the brand name anyway because words mean what they mean in practice, I really do think that the thing that made middle Yudkowsky (circa 2005–2013) world-changingly valuable was his explanation of there being objective laws of thought (as exemplified by the “Technical Explanation”, “The Botttom Line”, or “The Second Law of Thermodynamics, and Engines of Cognition”), so when I see the brand name being used to market a particular set of discourse norms without a clear explanation of how these norms are derived from the law, that bothers me enough to quickly write an essay or two about it, even though this is probably not a great use of my time or community-drama-instigating budgets in the year 2023.
so when I see the brand name being used to market a particular set of discourse norms without a clear explanation of how these norms are derived from the law, that bothers me enough to quickly write an essay or two about it
Seems great to me! I share your intuition that Goodwill seems a bit odd to include. I think it’s right to push back on proposed norms like these and talk about how justified they are, and I hope my list can be the start of a conversation like that rather than the end.
I do have an intuition that Goodwill, or something similar to Goodwill, plays an important role in the vast majority of human discourse that reliably produces truth. But I’m not sure why; if I knew very crisply what was going on here, maybe I could reduce it to other rules that are simpler and more universal.
To be clear, I endorse you doing that, but I would like you to do it without sleight-of-hand-frame-control.
(I do agree you could probably have written the second half of the post without relying on the first half’s structure, but, that’s not what you did)
I have on my todo list to write up a post that’s like “hey guys here is an explanation of Frame Control/Manipulation that is more rigorous and more neutrally worded than Aella’s post about it, and here’s why I think we should have a habit of noticing it.”.
And then, maybe afterwards, a post going: “Hey, I think ‘notice your own frame control, and be a bit careful about it’ should graduate to a thing you are obligated to learn, as a good LW citizen. What do people think of that?”, and get some sense of how The People think about it. And, depending on how that goes, maybe it becomes an actual LW norm.
I haven’t done that and doesn’t seem fair to rush it or assume how that’ll play out, so, currently this is more of a suggestion that I think you should probably agree to on your own terms rather than something I’m enforcing as a moderator, but, flagging that that’s a longer term agenda of mine.
In your view, is there an important difference between frame control, and the author having a particular frame that they use in a particular essay?
I’m proud of this blog post. I think it’s a good blog post that clearly explains my ideas in a way that’s engaging to read. If someone wants to talk about my motivations for writing this post and why I chose the analogies I did, I’m happy to have that discussion in the comment section, like we’re doing now.
But it seems to me that a blog post that talked about my objections to Bensinger’s Goodwill element, without first explaining the “motors” and “martial arts” analogies as illustrations of how I’m thinking about the topic, would be worse than this post, primarily because it would be less faithful to how I’m thinking about the topic, but also because it would just be less interesting to read.
If someone thinks my choice of analogies (or “frames”; I’m not sure if there’s a specific definition of “frame” I’m supposed to be familar with in this context) is misleading for some specific reason, they’re welcome to argue that in the comment section. So far, you have not persuaded me that I should have made any different writing choices.
a thing you are obligated to learn, as a good LW citizen
I mean, it’s your website. (Or maybe it’s Oli’s or Vaniver’s website? I’m not keeping track of the power structure.) If think you can obligate your users to learn something in order to maintain some “good citizen” status, that’s definitely a thing you can try to do.
For myself, I use this website because I’m interested in the study of human rationality (as that study was articulated by Yudkowsky in 2007–2009), and this seems like a pretty good website for posting my thoughts and reading other people’s thoughts about that topic, and also because I have a long history here (and therefore a large investment in the local jargon, &c.).
I consider myself to have obligations under the moral law to think and write clearly. I do not consider myself to have any obligations whatsoever “as a good LW citizen”.
a post that’s like “hey guys here is an explanation of Frame Control/Manipulation that is more rigorous and more neutrally worded than Aella’s post about it, and here’s why I think we should have a habit of noticing it.”.
That sounds like a good blog post! I eagerly look forward to reading it.
In your view, is there an important difference between frame control, and the author having a particular frame that they use in a particular essay?
Yep!
Distinctions in Frame Control
I’m still working through this, which is part of why the post isn’t written up yet. I’m also not sure if I’m actually going to use the phrase ‘frame control’ because it might just be too easy to weaponize in a way that makes it more unhelpful than helpful. (i.e. the concept I have in mind here is something it makes to have the norm of ‘notice when you do it, and be careful with it’, not ‘don’t do it ever’)
But, here are my current thoughts on how I currently carve up the space here:
having a strongly held/presented frame, such as by speaking confidently/authoritatively (which many people who don’t hold their own frames very strongly sometimes find disorienting)
having an insistently held frame (where when someone tries to say/imply ‘hey, my frame is X’ you’re like ‘no, the frame is Y’ and if they’re like ‘no, it’s X’)
frame manipulation (where you change someone else’s frame in a subtle way without them noticing, i.e. presenting a set of assumptions in a way that aren’t natural to question, or equivocating on definitions of words in ways that change what sort of questions to think about without people noticing you’ve done so)
#2, #3 and #4 can be mixed and matched.
The places where people tend to use the word ‘frame control’ most often refer to #3 and #4, frame-manipulation and frame-insistence. I’m a bit confused about how to think about ‘strong frames’ – I think there’s nothing inherently wrong with them, but if Alice is ‘weaker willed’ than Bob, she may end up adopting his frame in ways that subtly hurt her. This isn’t that different from, like, some people being physically bigger and more likely to accidentally hurt a smaller person. I wouldn’t want society to punish people for happening-to-be-big, but it feels useful to at least notice ‘bigness privilege’ sometimes.
That said, strongly held frames that are also manipulative or insistent can be pretty hard for many people to be resilient against, and I think it’s worth noticing that.
Re: this particular post
This post felt like (mild) frame manipulation to me, here:
Contemplating these basics, it should be clear that there’s just not going to be anything like a unique style of “rationalist discourse”, any more than there is a unique “physicist motor.”
Where I think this objection only quite makes sense if you’re considering “rationalist discourse” to be “rationalist-as-scientist-studying-laws”, instead of “applied-rationalist”.
A thing that would have fixed it is noting “Duncan seems to be using a different definition of ‘rationalist’ and ‘basics’ here. But I think those definitions are less useful because [reasons].
(Note: I don’t think you actually need the ‘frame’ frame, or the ‘frame control’ frame, to object to your post on these grounds. Equivocation / changing definitions also just seems, like, ‘deceptive’)
I do want to flag: I also think Duncan’s ‘The Basics of Rationalist Discourse’ is also somewhat frame-manipulative while also being somewhat strongly held, by my definition here. I went out of my way to try and counter-the-frame-control in my curation notice. (I have some complicated thoughts on whether this was fine, whether it was more or less fine than my complaint here, but it’d take awhile more to put them into legible form)
I’m definitely doing #2. I can see your case that the paragraph starting with “But there’s a reason for that” is doing #4. But … I’m not convinced that this kind of “frame manipulation” is particularly bad?
If someone is unhappy with the post’s attempt to “grab the frame” (by acting as if my conception of rationalist is the correct one), I’m happy to explain why I did that in the comments. Do I have to disclaim it in the post? That just seems like it would be worse writing.
I think in isolation it wouldn’t be particularly bad, no. I think it’d rise to the level of ‘definitely better to avoid’ (given [probably?] shared assumptions about truthseeking and honesty), but, it’s within the set of mistakes I think are fairly normal to make.
I feel like it is part of a broader pattern that (I think probably) adds up to something noticeably bad, but it’d take me awhile of active effort to find all the things that felt off to me and figure out if I endorse criticizing it as a whole.
(So, like, for now I’m not trying to make a strong argument that there’s a particular thing that’s wrong, but, like, I think you have enough self-knowledge to notice ‘yeah something is off in a sticky way here’ and figure it out yourself. ((But, as previously stated, I don’t have a strong belief that this makes sense to be your priority atm)))
A thing that would have fixed it is noting “Duncan seems to be using a different definition of ‘rationalist’ and ‘basics’ here. But I think those definitions are less useful because [reasons].
Oh, also to clarify, in my current view, you don’t need to tack on the ‘because [reasons]’ to avoid it being frame manipulation. Simply noting that you think it makes more sense to use a different definition is enough to dispel the sleight of hand feeling. (Although listing reasons may make it more persuasive that people use this definition rather than another one)
Are these two things not intimately connected? Should we not study the laws of rationality in the course of training to think rationally (indeed, in the course of determining what it means to think rationally, and determining how to train to think rationally)? And what is the point of studying the laws of rationality, if not to apply them?
I certainly do think they’re connected, but, they are still distinct concepts, and I think part of the reason Zack is focused on “rationalists as students of the laws of rationality” vs “applicants thereof” is that a community of law-studiers should behave differently. (as I understand it, pure math and applied math are different and people make a big deal about it??)
(and, to be clear, I think this is a pretty interesting consideration I hadn’t been thinking of lately. I appreciate Zack bringing it up, just want him to not be slight-of-handy about it)
Hmm, but I don’t think that rationality is actually analogous to math, in this respect. I think that the intimate connection between learning and applying rationality is, actually, a core property of rationality as a domain, as distinct from domains like math. Any disconnect between study and application threatens to undermine both!
It looks like this post is resting on defining “rationalist” as “one who studies the laws of rationality”, as opposed “someone training to think rationally”, but, hasn’t really acknowledged that it’s using this definition (when I think Duncan and Robby’s posts seem pointed more at the latter definition)
(Actually, looking more, I think this post sort of equivocates between the two, without noting that it’s done so).
I’m not 100% sure I’ve parsed this right, but, this looks at first glance like the sort of language trick that you (Zack) are often (rightfully) annoyed at.
(I think it’s a reasonable conversational move to point out someone else’s definition of a word isn’t the only valid definition, and pointing out their frame isn’t the only valid frame. But if you’re doing that it’s better to do that explicitly)
I’ll agree that the “physicist motors” analogy in particular rests on the “one who studies” definition, although I think a lot of the points I make in this essay don’t particularly depend on the analogy and could easily be written up separately.
I guess you could view the “foreign policy” motivating this post as being driven by two motives: first, I’d rather not waste precious time (in the year 2023, when a lot of us have more important things to do) fighting over the “rationalist” brand name; if someone else who also cares about thinking well, thinks that I’m going about everything all wrong, I think it’s fine that we just have our own separate dojos, Archipelago-style. That’s why the post emphasizes that there are many types of motors and many types of marital arts.
But secondly, insofar as I’m unfortunately stuck fighting over the brand name anyway because words mean what they mean in practice, I really do think that the thing that made middle Yudkowsky (circa 2005–2013) world-changingly valuable was his explanation of there being objective laws of thought (as exemplified by the “Technical Explanation”, “The Botttom Line”, or “The Second Law of Thermodynamics, and Engines of Cognition”), so when I see the brand name being used to market a particular set of discourse norms without a clear explanation of how these norms are derived from the law, that bothers me enough to quickly write an essay or two about it, even though this is probably not a great use of my time or community-drama-instigating budgets in the year 2023.
Seems great to me! I share your intuition that Goodwill seems a bit odd to include. I think it’s right to push back on proposed norms like these and talk about how justified they are, and I hope my list can be the start of a conversation like that rather than the end.
I do have an intuition that Goodwill, or something similar to Goodwill, plays an important role in the vast majority of human discourse that reliably produces truth. But I’m not sure why; if I knew very crisply what was going on here, maybe I could reduce it to other rules that are simpler and more universal.
To be clear, I endorse you doing that, but I would like you to do it without sleight-of-hand-frame-control.
(I do agree you could probably have written the second half of the post without relying on the first half’s structure, but, that’s not what you did)
I have on my todo list to write up a post that’s like “hey guys here is an explanation of Frame Control/Manipulation that is more rigorous and more neutrally worded than Aella’s post about it, and here’s why I think we should have a habit of noticing it.”.
And then, maybe afterwards, a post going: “Hey, I think ‘notice your own frame control, and be a bit careful about it’ should graduate to a thing you are obligated to learn, as a good LW citizen. What do people think of that?”, and get some sense of how The People think about it. And, depending on how that goes, maybe it becomes an actual LW norm.
I haven’t done that and doesn’t seem fair to rush it or assume how that’ll play out, so, currently this is more of a suggestion that I think you should probably agree to on your own terms rather than something I’m enforcing as a moderator, but, flagging that that’s a longer term agenda of mine.
In your view, is there an important difference between frame control, and the author having a particular frame that they use in a particular essay?
I’m proud of this blog post. I think it’s a good blog post that clearly explains my ideas in a way that’s engaging to read. If someone wants to talk about my motivations for writing this post and why I chose the analogies I did, I’m happy to have that discussion in the comment section, like we’re doing now.
But it seems to me that a blog post that talked about my objections to Bensinger’s Goodwill element, without first explaining the “motors” and “martial arts” analogies as illustrations of how I’m thinking about the topic, would be worse than this post, primarily because it would be less faithful to how I’m thinking about the topic, but also because it would just be less interesting to read.
If someone thinks my choice of analogies (or “frames”; I’m not sure if there’s a specific definition of “frame” I’m supposed to be familar with in this context) is misleading for some specific reason, they’re welcome to argue that in the comment section. So far, you have not persuaded me that I should have made any different writing choices.
I mean, it’s your website. (Or maybe it’s Oli’s or Vaniver’s website? I’m not keeping track of the power structure.) If think you can obligate your users to learn something in order to maintain some “good citizen” status, that’s definitely a thing you can try to do.
For myself, I use this website because I’m interested in the study of human rationality (as that study was articulated by Yudkowsky in 2007–2009), and this seems like a pretty good website for posting my thoughts and reading other people’s thoughts about that topic, and also because I have a long history here (and therefore a large investment in the local jargon, &c.).
I consider myself to have obligations under the moral law to think and write clearly. I do not consider myself to have any obligations whatsoever “as a good LW citizen”.
That sounds like a good blog post! I eagerly look forward to reading it.
Yep!
Distinctions in Frame Control
I’m still working through this, which is part of why the post isn’t written up yet. I’m also not sure if I’m actually going to use the phrase ‘frame control’ because it might just be too easy to weaponize in a way that makes it more unhelpful than helpful. (i.e. the concept I have in mind here is something it makes to have the norm of ‘notice when you do it, and be careful with it’, not ‘don’t do it ever’)
But, here are my current thoughts on how I currently carve up the space here:
having a frame, at all [i.e. set of ways to conceptualize a problem or solution-space or what questions to ask]
having a strongly held/presented frame, such as by speaking confidently/authoritatively (which many people who don’t hold their own frames very strongly sometimes find disorienting)
having an insistently held frame (where when someone tries to say/imply ‘hey, my frame is X’ you’re like ‘no, the frame is Y’ and if they’re like ‘no, it’s X’)
frame manipulation (where you change someone else’s frame in a subtle way without them noticing, i.e. presenting a set of assumptions in a way that aren’t natural to question, or equivocating on definitions of words in ways that change what sort of questions to think about without people noticing you’ve done so)
#2, #3 and #4 can be mixed and matched.
The places where people tend to use the word ‘frame control’ most often refer to #3 and #4, frame-manipulation and frame-insistence. I’m a bit confused about how to think about ‘strong frames’ – I think there’s nothing inherently wrong with them, but if Alice is ‘weaker willed’ than Bob, she may end up adopting his frame in ways that subtly hurt her. This isn’t that different from, like, some people being physically bigger and more likely to accidentally hurt a smaller person. I wouldn’t want society to punish people for happening-to-be-big, but it feels useful to at least notice ‘bigness privilege’ sometimes.
That said, strongly held frames that are also manipulative or insistent can be pretty hard for many people to be resilient against, and I think it’s worth noticing that.
Re: this particular post
This post felt like (mild) frame manipulation to me, here:
Where I think this objection only quite makes sense if you’re considering “rationalist discourse” to be “rationalist-as-scientist-studying-laws”, instead of “applied-rationalist”.
A thing that would have fixed it is noting “Duncan seems to be using a different definition of ‘rationalist’ and ‘basics’ here. But I think those definitions are less useful because [reasons].
(Note: I don’t think you actually need the ‘frame’ frame, or the ‘frame control’ frame, to object to your post on these grounds. Equivocation / changing definitions also just seems, like, ‘deceptive’)
I do want to flag: I also think Duncan’s ‘The Basics of Rationalist Discourse’ is also somewhat frame-manipulative while also being somewhat strongly held, by my definition here. I went out of my way to try and counter-the-frame-control in my curation notice. (I have some complicated thoughts on whether this was fine, whether it was more or less fine than my complaint here, but it’d take awhile more to put them into legible form)
I’m definitely doing #2. I can see your case that the paragraph starting with “But there’s a reason for that” is doing #4. But … I’m not convinced that this kind of “frame manipulation” is particularly bad?
If someone is unhappy with the post’s attempt to “grab the frame” (by acting as if my conception of rationalist is the correct one), I’m happy to explain why I did that in the comments. Do I have to disclaim it in the post? That just seems like it would be worse writing.
I think in isolation it wouldn’t be particularly bad, no. I think it’d rise to the level of ‘definitely better to avoid’ (given [probably?] shared assumptions about truthseeking and honesty), but, it’s within the set of mistakes I think are fairly normal to make.
I feel like it is part of a broader pattern that (I think probably) adds up to something noticeably bad, but it’d take me awhile of active effort to find all the things that felt off to me and figure out if I endorse criticizing it as a whole.
(So, like, for now I’m not trying to make a strong argument that there’s a particular thing that’s wrong, but, like, I think you have enough self-knowledge to notice ‘yeah something is off in a sticky way here’ and figure it out yourself. ((But, as previously stated, I don’t have a strong belief that this makes sense to be your priority atm)))
Oh, also to clarify, in my current view, you don’t need to tack on the ‘because [reasons]’ to avoid it being frame manipulation. Simply noting that you think it makes more sense to use a different definition is enough to dispel the sleight of hand feeling. (Although listing reasons may make it more persuasive that people use this definition rather than another one)
Are these two things not intimately connected? Should we not study the laws of rationality in the course of training to think rationally (indeed, in the course of determining what it means to think rationally, and determining how to train to think rationally)? And what is the point of studying the laws of rationality, if not to apply them?
I certainly do think they’re connected, but, they are still distinct concepts, and I think part of the reason Zack is focused on “rationalists as students of the laws of rationality” vs “applicants thereof” is that a community of law-studiers should behave differently. (as I understand it, pure math and applied math are different and people make a big deal about it??)
(and, to be clear, I think this is a pretty interesting consideration I hadn’t been thinking of lately. I appreciate Zack bringing it up, just want him to not be slight-of-handy about it)
Hmm, but I don’t think that rationality is actually analogous to math, in this respect. I think that the intimate connection between learning and applying rationality is, actually, a core property of rationality as a domain, as distinct from domains like math. Any disconnect between study and application threatens to undermine both!
The beauty of the subject!