Thanks for the post, I find this unique style really refreshing.
I would add to it that there’s even an “alignment problem” on the individual level. A single human in different circumstances and at different times can have quite different, sometimes incompatible values, preferences and priorities. And even at any given moment their values may be internally inconsistent and contradictory. So this problem exists on many different levels. We haven’t “solved ethics”, humanity disagrees about everything, even individual humans disagree with themselves, and now we’re suddenly racing towards a point where we need to give AI a definite idea of what is good & acceptable.
Yes, the individual and the collected are tightly coupled with short-term and long-term goals, which exist within individuals too. I think it’s interesting to think of yourself as a city, where you need to make systemic changes sometimes to enable individual flourishing.
I really think there is something to making alignment the actual goal of AI—but in a way where the paradoxical nature of alignment is acknowledged, so the AI is not looking for a “final solution” but is rather measuring the success of various strategies in lowering society’s (to return to the metaphor of the individual) cognitive dissonance.
Just to note your last paragraph reminds me of Stuart Russel’s approach to AI alignment in Human Compatible. And I agree this sounds like a reasonable starting point.
Thanks for the post, I find this unique style really refreshing.
I would add to it that there’s even an “alignment problem” on the individual level. A single human in different circumstances and at different times can have quite different, sometimes incompatible values, preferences and priorities. And even at any given moment their values may be internally inconsistent and contradictory. So this problem exists on many different levels. We haven’t “solved ethics”, humanity disagrees about everything, even individual humans disagree with themselves, and now we’re suddenly racing towards a point where we need to give AI a definite idea of what is good & acceptable.
Thanks, very astute point.
Yes, the individual and the collected are tightly coupled with short-term and long-term goals, which exist within individuals too. I think it’s interesting to think of yourself as a city, where you need to make systemic changes sometimes to enable individual flourishing.
I really think there is something to making alignment the actual goal of AI—but in a way where the paradoxical nature of alignment is acknowledged, so the AI is not looking for a “final solution” but is rather measuring the success of various strategies in lowering society’s (to return to the metaphor of the individual) cognitive dissonance.
Just to note your last paragraph reminds me of Stuart Russel’s approach to AI alignment in Human Compatible. And I agree this sounds like a reasonable starting point.
There’s a tiny possibility he may have influenced my thinking. I did spend 6 months editing him, among others for a documentary.