I’m not sure the distinction between “tool AI” and “agent AI” is that important when thinking of the existential risks of AI; as has already been said (by Holden Karnofsky himself!), a tool AI can be relatively easily turned into an agent AI.
We already have tools like Google search, but we also already have agents like scripts that check the status of servers and send mail under certain conditions, or the robot that cleans my living room. So when we get the technology for a tool AGI, we’ll have the technology for an agent AGI. And we won’t have less demand for agents than we already have now.
I’m not sure the distinction between “tool AI” and “agent AI” is that important when thinking of the existential risks of AI; as has already been said (by Holden Karnofsky himself!), a tool AI can be relatively easily turned into an agent AI.
We already have tools like Google search, but we also already have agents like scripts that check the status of servers and send mail under certain conditions, or the robot that cleans my living room. So when we get the technology for a tool AGI, we’ll have the technology for an agent AGI. And we won’t have less demand for agents than we already have now.