You might like to know: I debated erasing that part and the one that followed, thinking of you replying to it! :-D
But I figured hey, let’s have the chat. :-)
It seems to me that you can think about questions of alignment from a purely technical mindset…
Yep, I know it seems that way.
And I disagree. I think it maintains a confusion about what “alignment” is.
However, I’m less certain of this detail than I am about the overall picture. The part that has me say “We’re already in AI takeoff.” Which is why I debated erasing all the stuff about identity and first-person. It’s a subtle point that probably deserves its own separate post, if I ever care to write it. The rest stands on its own I think.
But! Setting that aside for a second:
To think of “questions of alignment from a purely technical mindset”, you need to call up an image of each of:
the AI
human values
some process by which these connect
But when you do this, you’re viewing them in third person. You have to call these images (visual or not) in your mind, and then you’re looking at them.
What the hell is this “human values” thing that’s separable from the “you” who’s looking?
The illusion that this is possible creates a gap that summons Goodhart. The distance between your subjective first-person experience and whatever concept of “human values” you see in third person is precisely what summons horror.
That’s the same gap that unFriendly egregores use to stupefy minds.
You can’t get around this by taking “subjectivity” or “consciousness” or whatever as yet another object that “humans care about”.
The only way I see to get around this is to recognize in immediate direct experience how your subjectivity — not as a concept, but as a direct experience — is in fact deeply inextricable from what you care about.
And that this is the foundation for all care.
When you tune a mind to correctly reflect this, you aren’t asking how this external AI aligns with “human values”. You’re asking how it synchs up with your subjective experience.
(Here minds get super squirrely. It’s way, way too easy to see “subjective experience” in third person.)
As you solve that, it becomes super transparent that sorting out that question is actually the same damn thing as asking how to be free of stupefaction, and how to be in clear and authentic connection with other human beings.
So, no. I don’t think you can solve this with a purely technical mindset. I think that perpetuates exactly the problem that this mindset would be trying to solve.
…but I could be wrong. I’m quite open to that.
So you believe them, feel unconcerned as you design your AI to detect and neutralize irrational hypercreatures, and then suddenly oh what happened, how could you ever have believed that crazy old thing before.
This part made me chuckle. :-)
I do think this is roughly how it works. It’s just that it happens via memetics first.
But overall, I agree in principle. If I’m wrong and it’s possible to orient to AI alignment as a purely technical problem, then yes, it’s possible to sidestep hypercreature hostility by kind of hitting them in an evolutionary blindspot.
When you tune a mind to correctly reflect this, you aren’t asking how this external AI aligns with “human values”. You’re asking how it synchs up with your subjective experience.
Any further detail you’d like to give on what constitutes “synching up with your subjective experience” (in the sense relevant to making an intelligence that produces plans that transform the world, without killing everyone)? :)
This is a koan-type meditative puzzle FWIW. A hint:
When you look outside and see a beautiful sky, you can admire it and think “Wow, that’s a beautiful sky.” But the knowing had to happen before the thought. What do you see when you attend to the level of knowing that comes before all thought?
That’s not a question to answer. It’s an invitation to look.
Not meaning to be obtuse here. This is the most direct I know how to be right now.
You might like to know: I debated erasing that part and the one that followed, thinking of you replying to it! :-D
But I figured hey, let’s have the chat. :-)
Yep, I know it seems that way.
And I disagree. I think it maintains a confusion about what “alignment” is.
However, I’m less certain of this detail than I am about the overall picture. The part that has me say “We’re already in AI takeoff.” Which is why I debated erasing all the stuff about identity and first-person. It’s a subtle point that probably deserves its own separate post, if I ever care to write it. The rest stands on its own I think.
But! Setting that aside for a second:
To think of “questions of alignment from a purely technical mindset”, you need to call up an image of each of:
the AI
human values
some process by which these connect
But when you do this, you’re viewing them in third person. You have to call these images (visual or not) in your mind, and then you’re looking at them.
What the hell is this “human values” thing that’s separable from the “you” who’s looking?
The illusion that this is possible creates a gap that summons Goodhart. The distance between your subjective first-person experience and whatever concept of “human values” you see in third person is precisely what summons horror.
That’s the same gap that unFriendly egregores use to stupefy minds.
You can’t get around this by taking “subjectivity” or “consciousness” or whatever as yet another object that “humans care about”.
The only way I see to get around this is to recognize in immediate direct experience how your subjectivity — not as a concept, but as a direct experience — is in fact deeply inextricable from what you care about.
And that this is the foundation for all care.
When you tune a mind to correctly reflect this, you aren’t asking how this external AI aligns with “human values”. You’re asking how it synchs up with your subjective experience.
(Here minds get super squirrely. It’s way, way too easy to see “subjective experience” in third person.)
As you solve that, it becomes super transparent that sorting out that question is actually the same damn thing as asking how to be free of stupefaction, and how to be in clear and authentic connection with other human beings.
So, no. I don’t think you can solve this with a purely technical mindset. I think that perpetuates exactly the problem that this mindset would be trying to solve.
…but I could be wrong. I’m quite open to that.
This part made me chuckle. :-)
I do think this is roughly how it works. It’s just that it happens via memetics first.
But overall, I agree in principle. If I’m wrong and it’s possible to orient to AI alignment as a purely technical problem, then yes, it’s possible to sidestep hypercreature hostility by kind of hitting them in an evolutionary blindspot.
Any further detail you’d like to give on what constitutes “synching up with your subjective experience” (in the sense relevant to making an intelligence that produces plans that transform the world, without killing everyone)? :)
Not at the moment. I might at some other time.
This is a koan-type meditative puzzle FWIW. A hint:
When you look outside and see a beautiful sky, you can admire it and think “Wow, that’s a beautiful sky.” But the knowing had to happen before the thought. What do you see when you attend to the level of knowing that comes before all thought?
That’s not a question to answer. It’s an invitation to look.
Not meaning to be obtuse here. This is the most direct I know how to be right now.
Ok thanks.