This is just the paralysis argument. (Maybe any sophisticated non-consequentialists will have to avoid this anyway. Maybe this shows that non-consequentialism is unappealing.)
[Edit after Buck’s reply: I think it’s weaker because most Anthropic employees aren’t causing the possible-deaths, just participating in a process that might cause deaths.]
This is just the paralysis argument. (Maybe any sophisticated non-consequentialists will have to avoid this anyway. Maybe this shows that non-consequentialism is unappealing.)
[Edit after Buck’s reply: I think it’s weaker because most Anthropic employees aren’t causing the possible-deaths, just participating in a process that might cause deaths.]
I think it’s a bit stronger than the usual paralysis argument in this case, but yeah.