Yeah, I think I remember hearing about ARC doing this a while ago too, and disliked it then, and similarly dislike it now. Suppose they make a misaligned model, and their control systems fail so that the model can spread very far. I expect their unconstrained misaligned model can do far more damage than their constrained possibly aligned ones if able to spread freely on the internet. Probably being an existential risk itself.
Edit: Man, I kinda dislike my past comment. Like listening to or watching a recording of yourself. But I stick with the lab-leak concern.
Yeah, I think I remember hearing about ARC doing this a while ago too, and disliked it then, and similarly dislike it now. Suppose they make a misaligned model, and their control systems fail so that the model can spread very far. I expect their unconstrained misaligned model can do far more damage than their constrained possibly aligned ones if able to spread freely on the internet. Probably being an existential risk itself.
Edit: Man, I kinda dislike my past comment. Like listening to or watching a recording of yourself. But I stick with the lab-leak concern.