I started out disagreeing with where I thought this comment was going, but I think ended up reasonably sold by the end.
I want to flag something like “in any ‘normal’ circumstances, avoiding an earth-sized or even nation-sized moral-catastrophe is, like, really important?” I think it’s… might actually be correct to actually do some amount of hang-wringing about that even if you know you’re ultimately going to have to make the tradeoff against it? (mostly out of a general worry about being too quick to steamroll your moral intuitions with math).
But, yeah the circumstances aren’t normal, and seems likely there’s at least some tradeoff here.
I am generally pleasantly surprised that AI welfare is one (at least one (relatively?) senior) Anthropic employee’s roadmap at all.
I wasn’t expecting it to be there at all. (Though I’m sort of surprised an Anthropic folk is publicly talking about AI welfare but still not explicitly extinction risk)
I started out disagreeing with where I thought this comment was going, but I think ended up reasonably sold by the end.
I want to flag something like “in any ‘normal’ circumstances, avoiding an earth-sized or even nation-sized moral-catastrophe is, like, really important?” I think it’s… might actually be correct to actually do some amount of hang-wringing about that even if you know you’re ultimately going to have to make the tradeoff against it? (mostly out of a general worry about being too quick to steamroll your moral intuitions with math).
But, yeah the circumstances aren’t normal, and seems likely there’s at least some tradeoff here.
I am generally pleasantly surprised that AI welfare is one (at least one (relatively?) senior) Anthropic employee’s roadmap at all.
I wasn’t expecting it to be there at all. (Though I’m sort of surprised an Anthropic folk is publicly talking about AI welfare but still not explicitly extinction risk)