On the inside, before you Look, the thing you’re about to Look at doesn’t look on the inside like “high-level cognitive content”. It looks like how things are. This ends up with me saying things that sound kind of crazy or nonsensical, but to me are obvious once I Look at them. (E.g., there are no objects. We create objects in order to think. Because language is suffused with object-ness, though, I don’t know of any coherent way of talking about this.)
Because we don’t instinctively see our intuitions as “intuitions”, we just see them as the world. When you look at a green cup, you don’t think of yourself as seeing a picture reconstructed in your visual cortex—although that is what you are seeing—you just see a green cup. You think, “Why, look, this cup is green,” not, “The picture in my visual cortex of this cup is green.”
And in the same way, when people argue over whether the falling tree makes a sound, or whether Pluto is a planet, they don’t see themselves as arguing over whether a categorization should be active in their neural networks. It seems like either the tree makes a sound, or not.
In the same post, Eliezer also wrote:
It takes a deliberate effort to visualize your brain from the outside—and then you still don’t see your actual brain; you imagine what you think is there, hopefully based on science, but regardless, you don’t have any direct access to neural network structures from introspection.
It sounds like Looking is a skill that lets someone have more introspective access to their own neural network structures. If this is a correct understanding, it seems perfectly compatible with LW’s current approach to ontology, or at least the approach laid out in Eliezer’s Sequences (with one caveat being that I think we should be careful/skeptical about whether someone purporting to be Looking is really introspecting parts of their neural network structures, or merely doing some form of epistemic wireheading). Do you agree?
It sounds like Looking is a skill that lets someone have more introspective access to their own neural network structures. If this is a correct understanding, it seems perfectly compatible with LW’s current approach to ontology, or at least the approach laid out in Eliezer’s Sequences (with one caveat being that I think we should be careful/skeptical about whether someone purporting to be Looking is really introspecting parts of their neural network structures, or merely doing some form of epistemic wireheading). Do you agree?
Hmm. I need to answer this in two pieces simultaneously:
The short and slightly deceptive answer is “Yes I agree.” A more careful answer: From within LW’s current approach to ontology, the restriction of Looking to that ontology works perfectly well, although there are some things (like what Eric S. Raymond refers to in Dancing With the Gods) that will at best make sense while remaining largely inaccessible.
Your very first sentence here presupposes the standard LW ontology: “It sounds like Looking is a skill that lets someone have more introspective access to their own neural network structures.” The structure of your question then goes on to ask about Looking’s compatibility with that ontology from within that ontology. The answer has to be “yes”, because the question makes sense within the ontology. This generates a “Get out of the car” problem. This isn’t a huge problem right here and now, but it will be a problem down the road when I start more explicitly pointing at some results of Looking at ontologies.
Hmm… So going back to the paragraph I was responding to:
On the inside, before you Look, the thing you’re about to Look at doesn’t look on the inside like “high-level cognitive content”. It looks like how things are. This ends up with me saying things that sound kind of crazy or nonsensical, but to me are obvious once I Look at them. (E.g., there are no objects. We create objects in order to think. Because language is suffused with object-ness, though, I don’t know of any coherent way of talking about this.)
Are you saying that LW’s approach to ontology has a different problem from this (which causes it to not be able to create an ontology that captures everything that’s important about Looking)? (In other words, this paragraph wasn’t meant to apply to LW; LW has a different problem.) Or is it something more like, LW’s approach appreciates “we create objects in order to think” on an intellectual level but not on a practical level?
Or is it something more like, LW’s approach appreciates “we create objects in order to think” on an intellectual level but not on a practical level?
That one.
Though to be clear, I’m not trying to talk specifically about the “there are no objects” thing exactly. I was using that as an example of something seen via Looking that I imagine sounds kind of crazy or nonsensical.
But I do mean that LW culture occurs to me as being subject to its ontology, and to the extent that there’s discussion of this, that discussion is pretty reliably done within that ontology. This gives the illusion of it being justified (when that’s actually just a consistency check) and makes the ontology’s blindspots incredibly difficult to point out.
I don’t think that sentence exactly presupposes the standard LW ontology. Rather, Wei_Dai is saying: “It currently looks to me as if this Looking stuff is compatible with standard LW ontology, and here’s what it looks like; if that’s wrong, please explain how”.
I have largely lost hope, though, that any of the Enlightened[1] will seriously attempt to explain how, rather than just continuing to tell us Unenlightened[2] folks that our ontology, or paperclip-maximizer-like brain subagents, or whatever, block us from understanding. Of course they may well be right; perhaps explaining what we’re wrong about really is futile because we just need to Get Out Of The Car but nothing they tell us will help us know what a “car” is or in what way we’re “in” one or how to “get out”, until enlightenment strikes and we see—sorry, See—for ourselves. That doesn’t stop it being frustrating, though. Still, I continue to harbour some hope that Valentine’s future articles may be, um, enlightening.
[1] I don’t mean to imply (1) that the people in question have achieved True Enlightenment, whatever that may be, or (2) that they think they have, or (3) that their alleged enlightenment is real. Though in any given case, any subset of those things might be the case.
[2] I don’t mean to imply (1) that the people in question are genuinely lacking some valuable insight or ability, or (2) that any particular other person thinks they have, or (3) that any particular other person thinks them inferior if they have. Though in any given case, any subset of those things might be the case.
(One specific possibility relevant to those footnotes is worth being explicit about: it could be that the Enlightened have genuine insights that they have gained through their Enlightenment—but that some of the Unenlightened have some of the same insights too, but it’s difficult to recognize that one insight is the same as the other. E.g., one thing Enlightened people sometimes report is a discovery that in some sense the “self” is unreal; some philosophers, neuroscientists, etc., have reached somewhat similar conclusions by very different routes; perhaps these are all correct discoveries of a single underlying truth, expressed in different terms.)
I’m confused. You seem to be expressing frustration at not getting a clear explanation of how Looking is incompatible with standard LW ontology, but Val just said that it is compatible with it?
Of course they may well be right; perhaps explaining what we’re wrong about really is futile because we just need to Get Out Of The Car but nothing they tell us will help us know what a “car” is or in what way we’re “in” one or how to “get out”,
… my post was an attempt to explain some of exactly that? A “car” is any belief or conceptual structure that you’re fused with; how you get out of it depends on the exact nature of the belief. For things like simple emotional reactions, just managing to distract yourself may be enough; for deeper conceptual structures, developing skill at being able to break down your cognitive processes into more basic building blocks is typically required.
I don’t think Valentine did quite say that (his notion of) Looking is compatible with standard LW ontology. He speaks of “the restriction of Looking to that ontology” and indicates that from within the standard LW ontology other things will “remain largely inaccessible”. He says that what Wei_Dai is saying “presupposes the standard LW ontology” and that this produces a “Get out of the car” problem. (While, yes, conceding that within that ontology “yes, it’s compatible” is the best available answer.)
I agree that your post is an attempt to explain those things. (And my slightly snarky comments about what “the Enlightened” are and aren’t willing to do was—I should have been explicit about this, sorry—not meant to apply to you: your clarity and explicitness on this stuff is extremely welcome.) But my impression is that, while Valentine has expressed approval of your post and said that he feels understood and so forth, he thinks there are important aspects of Looking/enlightenment/kensho/… that it doesn’t (and maybe can’t) cover.
Obvious disclaimer: I am not Valentine, and I may very well be misunderstanding him.
But my impression is that, while Valentine has expressed approval of your post and said that he feels understood and so forth, he thinks there are important aspects of Looking/enlightenment/kensho/… that it doesn’t (and maybe can’t) cover.
Doesn’t: yes, for sure.
Can’t: mmm, maybe? I expect that by the end of the sequence I’m writing, we’ll return to Kaj’s interpretation of Looking and basically just use it as a given — but it’ll mean something slightly different. Right now, I expect that if we just assume Kaj’s interpretation, we’re going to encounter a logjam when we apply Looking to the favored LW ontology, and the social web will have a kind of allergic reaction to the logjam that prevents collective understanding of where it came from. Once we collectively understand the structure of that whole process, we can smash face-first into the logjam, notice the confusion that results, and then make some meaningful progress on making our epistemic methods up to tackling serious meta-ontological challenges. At that point I think it’ll be just fine to say “Yep, we can think of Looking as compatible with the standard LW ontology.” Just not before.
I have largely lost hope, though, that any of the Enlightened[1] will seriously attempt to explain how, rather than just continuing to tell us Unenlightened[2] folks that our ontology, or paperclip-maximizer-like brain subagents, or whatever, block us from understanding.
I really am trying. When I talk about paperclip-maximizer-like subagents or ontological self-reference, it’s not my intent to say “You can’t understand because of XYZ.” I’m trying to say something more like, “I’d like you to notice the structure of XYZ and how it interferes with understanding, so that you notice and understand XYZ’s influence while we talk about the thing.”
Right now there’s too large of an inferential gap for me to answer the “how” question directly, and I can see specific ways in which my trying will just generate confusion, because of XYZs. But I really am trying to get there. It’s just going to take me a little while.
One specific possibility relevant to those footnotes is worth being explicit about: it could be that the Enlightened have genuine insights that they have gained through their Enlightenment—but that some of the Unenlightened have some of the same insights too, but it’s difficult to recognize that one insight is the same as the other.
Meta: Okay, I’m super confused what just happened. The webpage refreshed before I submitted my reply and from what I could tell just erased it. Then I wrote this one, submitted it, and the one I had thought was erased appeared as though I’d posted it.
I have largely lost hope, though, that any of the Enlightened[1] will seriously attempt to explain how, rather than just continuing to tell us Unenlightened[2] folks that our ontology, or paperclip-maximizer-like brain subagents, or whatever, block us from understanding.
I really am sincerely trying. In this case there’s a pretty epic inferential gap, and I’m working on bridging that gap… and it requires first talking about paperclip-maximizing-like mechanisms and illusions created by self-reference within ontologies that one is subject to. Then I can point at the Gödelian loophole, and we can watch our minds do summersaults, and we’ll recognize the summersaults and can step back and talk coherently about what the existence of the ontological wormhole might mean for epistemology.
Or at least that’s the plan.
And… I recognize it’s frustrating in the middle. And if I were more clever and/or more knowledgeable, I might have seen a way to make it less frustrating. I’d rather not create that experience for y’all.
FWIW, I don’t think the Unenlightened[2] can’t understand where I’m going. I just need some conceptual structures, like the social web thing, to make where I’m going even possible to say — at least given my current skill with expressing this stuff.
Still, I continue to harbour some hope that Valentine’s future articles may be, um, enlightening.
This sounds very familiar. To quote from How An Algorithm Feels From Inside:
In the same post, Eliezer also wrote:
It sounds like Looking is a skill that lets someone have more introspective access to their own neural network structures. If this is a correct understanding, it seems perfectly compatible with LW’s current approach to ontology, or at least the approach laid out in Eliezer’s Sequences (with one caveat being that I think we should be careful/skeptical about whether someone purporting to be Looking is really introspecting parts of their neural network structures, or merely doing some form of epistemic wireheading). Do you agree?
Hmm. I need to answer this in two pieces simultaneously:
The short and slightly deceptive answer is “Yes I agree.” A more careful answer: From within LW’s current approach to ontology, the restriction of Looking to that ontology works perfectly well, although there are some things (like what Eric S. Raymond refers to in Dancing With the Gods) that will at best make sense while remaining largely inaccessible.
Your very first sentence here presupposes the standard LW ontology: “It sounds like Looking is a skill that lets someone have more introspective access to their own neural network structures.” The structure of your question then goes on to ask about Looking’s compatibility with that ontology from within that ontology. The answer has to be “yes”, because the question makes sense within the ontology. This generates a “Get out of the car” problem. This isn’t a huge problem right here and now, but it will be a problem down the road when I start more explicitly pointing at some results of Looking at ontologies.
Hmm… So going back to the paragraph I was responding to:
Are you saying that LW’s approach to ontology has a different problem from this (which causes it to not be able to create an ontology that captures everything that’s important about Looking)? (In other words, this paragraph wasn’t meant to apply to LW; LW has a different problem.) Or is it something more like, LW’s approach appreciates “we create objects in order to think” on an intellectual level but not on a practical level?
That one.
Though to be clear, I’m not trying to talk specifically about the “there are no objects” thing exactly. I was using that as an example of something seen via Looking that I imagine sounds kind of crazy or nonsensical.
But I do mean that LW culture occurs to me as being subject to its ontology, and to the extent that there’s discussion of this, that discussion is pretty reliably done within that ontology. This gives the illusion of it being justified (when that’s actually just a consistency check) and makes the ontology’s blindspots incredibly difficult to point out.
I don’t think that sentence exactly presupposes the standard LW ontology. Rather, Wei_Dai is saying: “It currently looks to me as if this Looking stuff is compatible with standard LW ontology, and here’s what it looks like; if that’s wrong, please explain how”.
I have largely lost hope, though, that any of the Enlightened[1] will seriously attempt to explain how, rather than just continuing to tell us Unenlightened[2] folks that our ontology, or paperclip-maximizer-like brain subagents, or whatever, block us from understanding. Of course they may well be right; perhaps explaining what we’re wrong about really is futile because we just need to Get Out Of The Car but nothing they tell us will help us know what a “car” is or in what way we’re “in” one or how to “get out”, until enlightenment strikes and we see—sorry, See—for ourselves. That doesn’t stop it being frustrating, though. Still, I continue to harbour some hope that Valentine’s future articles may be, um, enlightening.
[1] I don’t mean to imply (1) that the people in question have achieved True Enlightenment, whatever that may be, or (2) that they think they have, or (3) that their alleged enlightenment is real. Though in any given case, any subset of those things might be the case.
[2] I don’t mean to imply (1) that the people in question are genuinely lacking some valuable insight or ability, or (2) that any particular other person thinks they have, or (3) that any particular other person thinks them inferior if they have. Though in any given case, any subset of those things might be the case.
(One specific possibility relevant to those footnotes is worth being explicit about: it could be that the Enlightened have genuine insights that they have gained through their Enlightenment—but that some of the Unenlightened have some of the same insights too, but it’s difficult to recognize that one insight is the same as the other. E.g., one thing Enlightened people sometimes report is a discovery that in some sense the “self” is unreal; some philosophers, neuroscientists, etc., have reached somewhat similar conclusions by very different routes; perhaps these are all correct discoveries of a single underlying truth, expressed in different terms.)
I’m confused. You seem to be expressing frustration at not getting a clear explanation of how Looking is incompatible with standard LW ontology, but Val just said that it is compatible with it?
… my post was an attempt to explain some of exactly that? A “car” is any belief or conceptual structure that you’re fused with; how you get out of it depends on the exact nature of the belief. For things like simple emotional reactions, just managing to distract yourself may be enough; for deeper conceptual structures, developing skill at being able to break down your cognitive processes into more basic building blocks is typically required.
I don’t think Valentine did quite say that (his notion of) Looking is compatible with standard LW ontology. He speaks of “the restriction of Looking to that ontology” and indicates that from within the standard LW ontology other things will “remain largely inaccessible”. He says that what Wei_Dai is saying “presupposes the standard LW ontology” and that this produces a “Get out of the car” problem. (While, yes, conceding that within that ontology “yes, it’s compatible” is the best available answer.)
I agree that your post is an attempt to explain those things. (And my slightly snarky comments about what “the Enlightened” are and aren’t willing to do was—I should have been explicit about this, sorry—not meant to apply to you: your clarity and explicitness on this stuff is extremely welcome.) But my impression is that, while Valentine has expressed approval of your post and said that he feels understood and so forth, he thinks there are important aspects of Looking/enlightenment/kensho/… that it doesn’t (and maybe can’t) cover.
Obvious disclaimer: I am not Valentine, and I may very well be misunderstanding him.
Doesn’t: yes, for sure.
Can’t: mmm, maybe? I expect that by the end of the sequence I’m writing, we’ll return to Kaj’s interpretation of Looking and basically just use it as a given — but it’ll mean something slightly different. Right now, I expect that if we just assume Kaj’s interpretation, we’re going to encounter a logjam when we apply Looking to the favored LW ontology, and the social web will have a kind of allergic reaction to the logjam that prevents collective understanding of where it came from. Once we collectively understand the structure of that whole process, we can smash face-first into the logjam, notice the confusion that results, and then make some meaningful progress on making our epistemic methods up to tackling serious meta-ontological challenges. At that point I think it’ll be just fine to say “Yep, we can think of Looking as compatible with the standard LW ontology.” Just not before.
Interesting. Let’s see what the sequence holds...
Got it. Apology accepted and appreciated. :)
I really am trying. When I talk about paperclip-maximizer-like subagents or ontological self-reference, it’s not my intent to say “You can’t understand because of XYZ.” I’m trying to say something more like, “I’d like you to notice the structure of XYZ and how it interferes with understanding, so that you notice and understand XYZ’s influence while we talk about the thing.”
Right now there’s too large of an inferential gap for me to answer the “how” question directly, and I can see specific ways in which my trying will just generate confusion, because of XYZs. But I really am trying to get there. It’s just going to take me a little while.
Strong agreement.
Meta: Okay, I’m super confused what just happened. The webpage refreshed before I submitted my reply and from what I could tell just erased it. Then I wrote this one, submitted it, and the one I had thought was erased appeared as though I’d posted it.
(And also, I can’t erase either one…?)
I really am sincerely trying. In this case there’s a pretty epic inferential gap, and I’m working on bridging that gap… and it requires first talking about paperclip-maximizing-like mechanisms and illusions created by self-reference within ontologies that one is subject to. Then I can point at the Gödelian loophole, and we can watch our minds do summersaults, and we’ll recognize the summersaults and can step back and talk coherently about what the existence of the ontological wormhole might mean for epistemology.
Or at least that’s the plan.
And… I recognize it’s frustrating in the middle. And if I were more clever and/or more knowledgeable, I might have seen a way to make it less frustrating. I’d rather not create that experience for y’all.
FWIW, I don’t think the Unenlightened[2] can’t understand where I’m going. I just need some conceptual structures, like the social web thing, to make where I’m going even possible to say — at least given my current skill with expressing this stuff.
Ha! :-)
I hope so too.