First, there is a lot of work in the “alignment community” that involves (for example) decision theory or open-source-game-theory or acausal trade, and I haven’t found any of it helpful for what I personally think about (which I’d like to think is “directly attacking the heart of the problem”, but others may judge for themselves when my upcoming post series comes out!).
I guess I see this subset of work as consistent with the hypothesis “some people have been nerd-sniped!”. But it’s also consistent with “some people have reasonable beliefs and I don’t share them, or maybe I haven’t bothered to understand them”. So I’m a bit loath to go around criticizing them, without putting more work into it. But still, this is a semi-endorsement of one of the things you’re saying.
Second, my understanding of MIRI (as an outsider, based purely on my vague recollection of their newsletters etc., and someone can correct me) is that (1) they have a group working on “better understand agent foundations”, and this group contains Abram and Scott, and they publish pretty much everything they’re doing, (2) they have a group working on undisclosed research projects, which are NOT “better understand agent foundations”, (3)
they have a couple “none of the above” people including Evan and Vanessa. So I’m confused that you seem to endorse what Abram and Scott are doing, but criticize agent foundations work at MIRI.
Like, maybe people “in the AI alignment community” are being nerd-sniped, and maybe MIRI had a historical role in how that happened, but I’m not sure there’s any actual MIRI employee right now who is doing nerd-sniped-type work, to the best of my limited understanding, unless we want to say Scott is, but you already said Scott is OK in your book.
(By the way, hot takes: I join you in finding some of Abram’s posts to be super helpful, and would throw Stuart Armstrong onto the “super helpful” list too, assuming he counts as “MIRI”. As for Scott: ironically, I find logical induction very useful when thinking about how to build AGI, and somewhat less useful when thinking about how to align it. :-P I didn’t get anything useful for my own thinking out of his Cartesian frames or finite factored sets, but as above, that could just be me; I’m very loath to criticize without doing more work, especially as they’re works in progress, I gather.)
Couple things:
First, there is a lot of work in the “alignment community” that involves (for example) decision theory or open-source-game-theory or acausal trade, and I haven’t found any of it helpful for what I personally think about (which I’d like to think is “directly attacking the heart of the problem”, but others may judge for themselves when my upcoming post series comes out!).
I guess I see this subset of work as consistent with the hypothesis “some people have been nerd-sniped!”. But it’s also consistent with “some people have reasonable beliefs and I don’t share them, or maybe I haven’t bothered to understand them”. So I’m a bit loath to go around criticizing them, without putting more work into it. But still, this is a semi-endorsement of one of the things you’re saying.
Second, my understanding of MIRI (as an outsider, based purely on my vague recollection of their newsletters etc., and someone can correct me) is that (1) they have a group working on “better understand agent foundations”, and this group contains Abram and Scott, and they publish pretty much everything they’re doing, (2) they have a group working on undisclosed research projects, which are NOT “better understand agent foundations”, (3) they have a couple “none of the above” people including Evan and Vanessa. So I’m confused that you seem to endorse what Abram and Scott are doing, but criticize agent foundations work at MIRI.
Like, maybe people “in the AI alignment community” are being nerd-sniped, and maybe MIRI had a historical role in how that happened, but I’m not sure there’s any actual MIRI employee right now who is doing nerd-sniped-type work, to the best of my limited understanding, unless we want to say Scott is, but you already said Scott is OK in your book.
(By the way, hot takes: I join you in finding some of Abram’s posts to be super helpful, and would throw Stuart Armstrong onto the “super helpful” list too, assuming he counts as “MIRI”. As for Scott: ironically, I find logical induction very useful when thinking about how to build AGI, and somewhat less useful when thinking about how to align it. :-P I didn’t get anything useful for my own thinking out of his Cartesian frames or finite factored sets, but as above, that could just be me; I’m very loath to criticize without doing more work, especially as they’re works in progress, I gather.)