I don’t think the paradigm is necessarily bad either, given enough alignment research. I think the point here is that these things are coming up clearly before we’ve given them enough alignment research.
Edit to add: Just reading through @Zvi’s latest AI update (AI #6: Agents of Change) and I will say he wrote a compelling argument for this being a good thing overall:
In terms of capabilities things quieted down. The biggest development is that people continue to furiously do their best to turn GPT-4 into a self-directed agent. At this point, I’m happy to see people working hard at this, so we don’t have an ‘agent overhang’ – if it is this easy to do, we want everything that can possibly go wrong to go wrong as quickly as possible, while the damage would be relatively contained.
then
If it is this easy to create an autonomous agent that can do major damage, much better to find that out now rather than wait when the damage would be worse or even existential. If such a program poses an existential risk now, then we live in a very very doomed world, and a close call as soon as possible would likely be our only hope of survival.
I don’t think the paradigm is necessarily bad either, given enough alignment research. I think the point here is that these things are coming up clearly before we’ve given them enough alignment research.
Edit to add: Just reading through @Zvi’s latest AI update (AI #6: Agents of Change) and I will say he wrote a compelling argument for this being a good thing overall:
then