I continue to have a feeling like “I don’t know how to interact with this without either falling into Ray’s ill-fitting framework and thus further confusing both of us, or completely failing to engage with his framework”. On the other hand I’m not sure I have tried, and perhaps I ought to try. Experience suggests that I will only succeed if I first ensure I don’t lose sight of whatever it is I can see from my current perspective, which is probably much of what I was up to in the previous thread, and that is probably why it seemed to you to “get weird”; I was halfway trying to say the relevant truths as I see them, and halfway trying to communicate with you, and I was not doing either of those things well as a result.
I do completely agree with everything @Duncan said in the other thread, and have said very similar things myself in the past, which perhaps I will post below as like supplementary reading or something. But I think it’s missing you because when I imagine you reading it I also imagine you responding with “But there really is a difference between trying to learn about things like ‘updating incrementally’ and trying to learn about things like ‘electromagnetism’.”
And I think I agree with (my imagined version of) you. Even if learning about things like “updating incrementally” requires learning about things like “electromagnetism” (which it does, at least in my own rationality framework and I suspect also in Eliezer’s, for whatever that’s worth), there is something different about setting out to study “updating incrementally” vs setting out to study “electromagnetism”.
What is different, according to me, is that studying electromagnetism by roughly naturalist methodology is way easier. It’s easier because the thing you’re trying to study is far less entangled with the tools you are using to study it. Things outside of the mind stay put in a way that things inside the mind do not. Which is why I made How To Observe Abstract Objects.
And it’s also most of why I made the course I talked about in the previous essay, and why I said that I wish everyone would start there. It’s almost the same set of tools, but it’s aimed at nature instead of at minds. This sequence is that course, but beefed up a bunch to contend with the unique difficulties of turning the cognitive tools on themselves, or something along those lines.
But I feel that I’m wandering away from communicating with you again. I think you approximately think that I should interweave my nature study course with this rationality sequence, because it would help people better understand what “naturalism” is, as a whole, and how to apply it in full generality. (Do you in fact think something like that?)
I think that if I were attempting to meet the goal of teaching people how to apply naturalism in full generality, I would probably agree with that. But in this particular sequence, I am not; I have learned over the years that I am capable of accomplishing at most one or two things at a time. Something about my cognitive and perceptual style, perhaps. And so even my “naturalist rationality sequence” is broken into three separate sequences: Intro to Naturalism, The Nuts and Bolts of Naturalism, and whatever I’ll end up calling the demonstration sequence. I do very much like the idea of eventually synthesizing all three sequences into a single concise guidebook, but I think that I simply am not smart enough to write that guidebook without first writing each piece of it. And then, even beyond that not-yet-existant “Naturalist Rationality” guidebook, there are other guidebooks I would love to create, such as something like “Naturalist approaches to finding traction in pre-theoretic fields”, which I expect will require just as much groundwork as I have so far put into “Naturalist approaches to mastering the basics of rationality”.
But I am again only half-talking to you, I see. Perhaps I should shut up for now, see if you have anything for me to listen to, and try again later.
A thing I found a bit confusing was, like, you list ‘what’s up with how things float in water’ as a thing you might study and get in contact with. Which contributed to a sense that, like, central examples of the things this course was meant to be about studying would include both things like ‘courage’ and things like ‘stuff floating on water’. You also listed ‘how to apply comparative advantage’. So I think I’m still confused about what you see the focus area of this sequence as.
I’m still feeling fairly surprised/confused that this conversation has felt so frame-clashy. I keep not being very surprised by most of the words you’re saying, but then somehow it still feels like there’s a stronger disconnect than I expect.
“But there really is a difference between trying to learn about things like ‘updating incrementally’ and trying to learn about things like ‘electromagnetism’.”
I maybe want to clarify, I don’t think there’s (necessarily) a difference between learning about things like ‘updating incrementally’ and learning about things like ‘electromagnetism’. My claim is “the average reader will think there is a difference (whether there is or not), and, if it’s an assumption running through your work that they are the same, or close-to-the-same in many ways, readers may be confused because they aren’t tracking that this is an assumption of yours.
Does that still feel ill-fitting-frame-y?
(I think there were other frame-disagreements at play here, but this was the part where I was feeling most confused about having apparently talked past each other or whatnot)
I just noticed that you have a post called “Noticing Frame Differences”, and I’m gonna go read it (in the next few days) in case that turns out to help.
Honestly I don’t expect it to much, it’s mostly just covering the basics of “frames exist”. (It does cover how, like, even pretty similar frames can be subtly different in ways that are difficult to track, but, like, that’s just enough to make me sadly aware of what’s happening but not necessarily be able to do anything about it)
Here is the “supplementary reading” I mentioned, (which perhaps belongs instead under Duncan’s comment in the other thread).
In an earlier draft, my opening sentence for this essay was, “Once I’ve identified a question that’s crucial to my story, it’s time to start observing the world—not so much to find an answer, as to position myself to ask better questions.” A beta reader highlighted “it’s time to start observing the world”, and said this about it (paraphrased): “I found this to be a confusing first sentence, because it makes it sound like this part of naturalism is about ‘observing the world’; but as I understand it, this part of naturalism is about ‘observing my reactions to the world’.”
To which I replied:
>I think part of what’s going on here is that we have… not quite different ontologies, I think, in the sense that we think different entities exist… but at least different conceptualizations of the world and minds at a pretty fundamental level.
>I don’t know how to talk about this well yet, despite an awful lot of attempts (like the “Intro To Naturalism” sequence), but I’ll take a stab at talking about it anyway.
>There is no such thing as “observing the world as distinct from observing my /reactions/ to the world.” Instead, the real distinction that actually exists is between “learning about the world by observing my reactions to it and knowing that what I’m observing are my reactions” and “learning about the world by observing my reactions to it and not knowing that what I’m observing are my reactions”. There’s no such thing as looking directly at a cup, in the way it’s most natural to imagine. All you can do is point your eyes toward a cup, and be aware of whatever you experience as a result.
>When I take a naturalist approach to cup observation, I “try to look at my experience as I direct my eyes toward cups”. This is not because it is actually possible to observe cups any more directly than that, but because framing it as a study of my own experience is a really powerful strategy mitigating the damage that map/territory conflations ordinarily cause to attempts to learn about things and solve problems.
>So when I’m “observing my experience of a cup”, I am actually doing my very best to observe the outside world. In fact I’m attempting to observe the outside world more accurately and precisely than I ordinarily could while merely “trying to observe cups”, because when I am aware that I’m observing my own experience, I am also aware that what exists in the world is distinct from my experience in various ways, even if I don’t understand what those ways are. My awareness of that distinction grants me much more freedom to hypothesize about other ways the actual world could be, and so whatever conclusions I draw about cups as a result contain less interference from my own reactions to cups. By observing my reactions to cups, I end up forming cup models that have less to do with me, and more to do with cups.
>Like imagine that you see a straw in a glass of water (for the first time). This seems weird to you, since straws are supposed to be straight, so you deicide to study it and try to figure out what’s going on.
>Imagine two different ways of approaching that study.
>The first way, you start with the question, “Why do straws bend when you stick them in water?”
>The second way, you start with the question, “Why do straws look to me as though they bend when you stick them in water?”
>In the second case you’re studying your reactions to straws in water, but you’re more likely to end up more quickly with models that involve how light works, because you recognized from the outset that “the straw itself bends” is not actually what you observed; what you actually observed is “the image of the straw that appears in my mind under these conditions bends”.
>A major thesis of mine is that “everything we cast our attention on is like the staw, to varying degrees”, and I tend to go “oh naturalism is an especially good idea for this person in particular” when I hear about their problem/curiosity/interest and it makes me think “ah yes, that right there sounds especially much like a bent straw”.
(Rationality, I claim, is absolutely chock full of bent straws, much more so than the vast majority of other fields of study.)
My beta reader found this response helpful, but they also were starting with a different set of thoughts than you are, I think. Still, I do wonder whether it does anything to bridge the gap between us.
I continue to have a feeling like “I don’t know how to interact with this without either falling into Ray’s ill-fitting framework and thus further confusing both of us, or completely failing to engage with his framework”. On the other hand I’m not sure I have tried, and perhaps I ought to try. Experience suggests that I will only succeed if I first ensure I don’t lose sight of whatever it is I can see from my current perspective, which is probably much of what I was up to in the previous thread, and that is probably why it seemed to you to “get weird”; I was halfway trying to say the relevant truths as I see them, and halfway trying to communicate with you, and I was not doing either of those things well as a result.
I do completely agree with everything @Duncan said in the other thread, and have said very similar things myself in the past, which perhaps I will post below as like supplementary reading or something. But I think it’s missing you because when I imagine you reading it I also imagine you responding with “But there really is a difference between trying to learn about things like ‘updating incrementally’ and trying to learn about things like ‘electromagnetism’.”
And I think I agree with (my imagined version of) you. Even if learning about things like “updating incrementally” requires learning about things like “electromagnetism” (which it does, at least in my own rationality framework and I suspect also in Eliezer’s, for whatever that’s worth), there is something different about setting out to study “updating incrementally” vs setting out to study “electromagnetism”.
What is different, according to me, is that studying electromagnetism by roughly naturalist methodology is way easier. It’s easier because the thing you’re trying to study is far less entangled with the tools you are using to study it. Things outside of the mind stay put in a way that things inside the mind do not. Which is why I made How To Observe Abstract Objects.
And it’s also most of why I made the course I talked about in the previous essay, and why I said that I wish everyone would start there. It’s almost the same set of tools, but it’s aimed at nature instead of at minds. This sequence is that course, but beefed up a bunch to contend with the unique difficulties of turning the cognitive tools on themselves, or something along those lines.
But I feel that I’m wandering away from communicating with you again. I think you approximately think that I should interweave my nature study course with this rationality sequence, because it would help people better understand what “naturalism” is, as a whole, and how to apply it in full generality. (Do you in fact think something like that?)
I think that if I were attempting to meet the goal of teaching people how to apply naturalism in full generality, I would probably agree with that. But in this particular sequence, I am not; I have learned over the years that I am capable of accomplishing at most one or two things at a time. Something about my cognitive and perceptual style, perhaps. And so even my “naturalist rationality sequence” is broken into three separate sequences: Intro to Naturalism, The Nuts and Bolts of Naturalism, and whatever I’ll end up calling the demonstration sequence. I do very much like the idea of eventually synthesizing all three sequences into a single concise guidebook, but I think that I simply am not smart enough to write that guidebook without first writing each piece of it. And then, even beyond that not-yet-existant “Naturalist Rationality” guidebook, there are other guidebooks I would love to create, such as something like “Naturalist approaches to finding traction in pre-theoretic fields”, which I expect will require just as much groundwork as I have so far put into “Naturalist approaches to mastering the basics of rationality”.
But I am again only half-talking to you, I see. Perhaps I should shut up for now, see if you have anything for me to listen to, and try again later.
That all honestly seems pretty reasonable.
A thing I found a bit confusing was, like, you list ‘what’s up with how things float in water’ as a thing you might study and get in contact with. Which contributed to a sense that, like, central examples of the things this course was meant to be about studying would include both things like ‘courage’ and things like ‘stuff floating on water’. You also listed ‘how to apply comparative advantage’. So I think I’m still confused about what you see the focus area of this sequence as.
I’m still feeling fairly surprised/confused that this conversation has felt so frame-clashy. I keep not being very surprised by most of the words you’re saying, but then somehow it still feels like there’s a stronger disconnect than I expect.
I maybe want to clarify, I don’t think there’s (necessarily) a difference between learning about things like ‘updating incrementally’ and learning about things like ‘electromagnetism’. My claim is “the average reader will think there is a difference (whether there is or not), and, if it’s an assumption running through your work that they are the same, or close-to-the-same in many ways, readers may be confused because they aren’t tracking that this is an assumption of yours.
Does that still feel ill-fitting-frame-y?
(I think there were other frame-disagreements at play here, but this was the part where I was feeling most confused about having apparently talked past each other or whatnot)
I just noticed that you have a post called “Noticing Frame Differences”, and I’m gonna go read it (in the next few days) in case that turns out to help.
Honestly I don’t expect it to much, it’s mostly just covering the basics of “frames exist”. (It does cover how, like, even pretty similar frames can be subtly different in ways that are difficult to track, but, like, that’s just enough to make me sadly aware of what’s happening but not necessarily be able to do anything about it)
Here is the “supplementary reading” I mentioned, (which perhaps belongs instead under Duncan’s comment in the other thread).
In an earlier draft, my opening sentence for this essay was, “Once I’ve identified a question that’s crucial to my story, it’s time to start observing the world—not so much to find an answer, as to position myself to ask better questions.” A beta reader highlighted “it’s time to start observing the world”, and said this about it (paraphrased): “I found this to be a confusing first sentence, because it makes it sound like this part of naturalism is about ‘observing the world’; but as I understand it, this part of naturalism is about ‘observing my reactions to the world’.”
To which I replied:
>I think part of what’s going on here is that we have… not quite different ontologies, I think, in the sense that we think different entities exist… but at least different conceptualizations of the world and minds at a pretty fundamental level.
>I don’t know how to talk about this well yet, despite an awful lot of attempts (like the “Intro To Naturalism” sequence), but I’ll take a stab at talking about it anyway.
>There is no such thing as “observing the world as distinct from observing my /reactions/ to the world.” Instead, the real distinction that actually exists is between “learning about the world by observing my reactions to it and knowing that what I’m observing are my reactions” and “learning about the world by observing my reactions to it and not knowing that what I’m observing are my reactions”. There’s no such thing as looking directly at a cup, in the way it’s most natural to imagine. All you can do is point your eyes toward a cup, and be aware of whatever you experience as a result.
>When I take a naturalist approach to cup observation, I “try to look at my experience as I direct my eyes toward cups”. This is not because it is actually possible to observe cups any more directly than that, but because framing it as a study of my own experience is a really powerful strategy mitigating the damage that map/territory conflations ordinarily cause to attempts to learn about things and solve problems.
>So when I’m “observing my experience of a cup”, I am actually doing my very best to observe the outside world. In fact I’m attempting to observe the outside world more accurately and precisely than I ordinarily could while merely “trying to observe cups”, because when I am aware that I’m observing my own experience, I am also aware that what exists in the world is distinct from my experience in various ways, even if I don’t understand what those ways are. My awareness of that distinction grants me much more freedom to hypothesize about other ways the actual world could be, and so whatever conclusions I draw about cups as a result contain less interference from my own reactions to cups. By observing my reactions to cups, I end up forming cup models that have less to do with me, and more to do with cups.
>Like imagine that you see a straw in a glass of water (for the first time). This seems weird to you, since straws are supposed to be straight, so you deicide to study it and try to figure out what’s going on.
>Imagine two different ways of approaching that study.
>The first way, you start with the question, “Why do straws bend when you stick them in water?”
>The second way, you start with the question, “Why do straws look to me as though they bend when you stick them in water?”
>In the second case you’re studying your reactions to straws in water, but you’re more likely to end up more quickly with models that involve how light works, because you recognized from the outset that “the straw itself bends” is not actually what you observed; what you actually observed is “the image of the straw that appears in my mind under these conditions bends”.
>A major thesis of mine is that “everything we cast our attention on is like the staw, to varying degrees”, and I tend to go “oh naturalism is an especially good idea for this person in particular” when I hear about their problem/curiosity/interest and it makes me think “ah yes, that right there sounds especially much like a bent straw”.
(Rationality, I claim, is absolutely chock full of bent straws, much more so than the vast majority of other fields of study.)
My beta reader found this response helpful, but they also were starting with a different set of thoughts than you are, I think. Still, I do wonder whether it does anything to bridge the gap between us.