The “meta-problem of consciousness”—a.k.a. “Why do people believe that there’s a hard problem of consciousness”—is about unraveling this chain of events.
I like this framing, similar to what Yudkowsky did for free will.
In terms of AGI, it seems to me that knowing whether or not AGI is conscious is an important thing to know, at least for the AGI’s sake. (Yeah I know—as if we don’t already have our hands full thinking about the impacts of AGI on humans!)
Honestly, my position is close to your imagined critic: worrying about the consciousness of the AGI seems somewhat less important that ensuring it doesn’t catastrophically influence all life. But then I wonder if this is the sort of false dichotomy that makes someone pass on a worthwile tradeoff...
That said, I’m tentatively feeling OK about what I wrote here—basically my proto-theory of the meta-problem of consciousness is some idiosyncratic mishmash of Michael Graziano’s theory, and Global Workspace theory, and Frankish-style illusionism. I’ll stay on the lookout for reasons to think that what I wrote there was wrong, but otherwise I think this topic is not high on my immediate priority list.
Reading quickly the linked book review, I’m slightly confused. You write that you don’t really know what the theory entails for AGI’s consciousness, so isn’t the actual application for the meta-problem of consciousness still wide open?
In particular, it seems pretty intimately tied to decision-making and motivation—for example, we are motivated to not suffer, and we make decisions that lead to not-suffering.
Surprising choice of examples, as my first thought was for the use of thinking about suffering in empathy, which sounds like a big missing part in many unaligned AGI.
So there’s the problem. How do we build a learned-from-scratch world model but shove little hardcoded pieces into it? How do we ensure that the hardcoded pieces wind up in the right place? How do we avoid the AGI ignoring the human model that we supplied it and instead building a parallel independent human model from scratch?
Building an actual LCDT agent looks like a particular case of this problem.
I think there’s a subfield of neuroscience called “consciousness studies”, where they talk a lot about how people formulate thoughts about themselves, etc. The obvious application is understanding consciousness, but I’m personally much more interested in whether it could help me think about The 1st-Person Problem. So I’m planning to dive into that sometime soon.
Thanks for letting me know about this field, it looks really exciting!
So I’m swallowing my impatience, and trying to really nail down dopamine supervised learning—keep talking to experts, keep filling in the gaps, keep searching for relevant evidence. And then I can feel better about building on it.
Kudos for doing that! This is indeed the best way of building a strong and plausible model that will stand the test of time, but sometimes it’s so hard to not go forward with it!
You write that you don’t really know what the theory entails for AGI’s consciousness, so isn’t the actual application for the meta-problem of consciousness still wide open?
I feel like I have a pretty good grasp on the solution to the meta-problem of consciousness but that I remain pretty confused and unsatisfied about the hard problem of consciousness. This is ironic because I was just saying that the hard problem should be relatively straightforward once you have the meta-problem nailed down. But “relatively straightforward” is still not trivial, especially given that I’m not an expert in the philosophy of consciousness and don’t want to spend the time to become one.
Surprising choice of examples, as my first thought was for the use of thinking about suffering in empathy, which sounds like a big missing part in many unaligned AGI.
Sure, but I think I was mentally lumping that under “social instincts”, which is a different section. Hmm, I guess I should have drawn an arrow between understanding suffering and understanding social instincts. They do seem to interact a bit.
(Fuller comment about the whole research agenda)
I like this framing, similar to what Yudkowsky did for free will.
Honestly, my position is close to your imagined critic: worrying about the consciousness of the AGI seems somewhat less important that ensuring it doesn’t catastrophically influence all life. But then I wonder if this is the sort of false dichotomy that makes someone pass on a worthwile tradeoff...
Reading quickly the linked book review, I’m slightly confused. You write that you don’t really know what the theory entails for AGI’s consciousness, so isn’t the actual application for the meta-problem of consciousness still wide open?
Surprising choice of examples, as my first thought was for the use of thinking about suffering in empathy, which sounds like a big missing part in many unaligned AGI.
Building an actual LCDT agent looks like a particular case of this problem.
Thanks for letting me know about this field, it looks really exciting!
Kudos for doing that! This is indeed the best way of building a strong and plausible model that will stand the test of time, but sometimes it’s so hard to not go forward with it!
I feel like I have a pretty good grasp on the solution to the meta-problem of consciousness but that I remain pretty confused and unsatisfied about the hard problem of consciousness. This is ironic because I was just saying that the hard problem should be relatively straightforward once you have the meta-problem nailed down. But “relatively straightforward” is still not trivial, especially given that I’m not an expert in the philosophy of consciousness and don’t want to spend the time to become one.
Sure, but I think I was mentally lumping that under “social instincts”, which is a different section. Hmm, I guess I should have drawn an arrow between understanding suffering and understanding social instincts. They do seem to interact a bit.