Thanks so much for writing this Seth! I so often get into conversations with people where I wished I had something like this post to refer them to. And now I do!
I really hope that you and Max’s ideas soon get the wider recognition that I think they deserve!
Thanks! A big part of my motivation for writing this was to try to direct more attention to Max’s excellent, detailed, and important work on personal intent alignment. And I wanted to understand the group communication/epistemics that have kept it from being as impactful as it seems to me to deserve.
People who think seriously about AGI alignment seem to still mostly be thinking that we’ll try to value align AGI, even though that’s nigh impossible in the near term. And they shouldn’t.
Thanks so much for writing this Seth! I so often get into conversations with people where I wished I had something like this post to refer them to. And now I do!
I really hope that you and Max’s ideas soon get the wider recognition that I think they deserve!
Thanks! A big part of my motivation for writing this was to try to direct more attention to Max’s excellent, detailed, and important work on personal intent alignment. And I wanted to understand the group communication/epistemics that have kept it from being as impactful as it seems to me to deserve.
People who think seriously about AGI alignment seem to still mostly be thinking that we’ll try to value align AGI, even though that’s nigh impossible in the near term. And they shouldn’t.