I think there’s probably value in being on an alignment team at a “capabilities” org, or even embedded in a capabilities team if the role itself doesn’t involve work that contributes to capabilities (either via first-order or second-order effects).
I think that the “in the room” argument might start to make sense when there’s actually a plan for alignment that’s in a sufficiently ready state to be operationalized. AFAICT nobody has such a plan yet. For that reason, I think maintaining & improving lines of communication is very important, but if I had to guess, I’d say you could get most of the anticipated benefit there without directly doing capabilities work.
Yes this does seem to be happening. It also appears to be unavoidable.
Our state of knowledge is nowhere near being able to guarantee that any AGI we develop will not kill us all. We are already developing AI that is superhuman in increasingly many aspects. Those who are actively working right now to bring the rest of the capabilities up to and above human levels obviously can’t be sufficiently concerned, or they would not be doing it.
I think there’s probably value in being on an alignment team at a “capabilities” org, or even embedded in a capabilities team if the role itself doesn’t involve work that contributes to capabilities (either via first-order or second-order effects).
I think that the “in the room” argument might start to make sense when there’s actually a plan for alignment that’s in a sufficiently ready state to be operationalized. AFAICT nobody has such a plan yet. For that reason, I think maintaining & improving lines of communication is very important, but if I had to guess, I’d say you could get most of the anticipated benefit there without directly doing capabilities work.
Yes this does seem to be happening. It also appears to be unavoidable.
Our state of knowledge is nowhere near being able to guarantee that any AGI we develop will not kill us all. We are already developing AI that is superhuman in increasingly many aspects. Those who are actively working right now to bring the rest of the capabilities up to and above human levels obviously can’t be sufficiently concerned, or they would not be doing it.