After some more reading, particularly the Drexler CAIS report, I realize I was more confused than I thought about Tool vs Agent AI. I think I’ve resolved it, but I’d appreciate feedback. Would the below be correct?
“Most sophisticated software behaves like both a Tool and an Agent, at different times. Google Maps reports possible routes like a tool, but it searches for paths to maximize a utility function like an agent. DeepMind might ultimately select the move that maximizes its winning probability, but it follows some set rules in how it frames and conducts this search. Only relatively simple programs could be called purely Tool: Atari Pong genuinely runs no search function. Any advanced AI, while it could be Tool in the sense of not taking actions that impact the outside world, is likely to be Agent in the sense of optimizing within bounds internally.”
Any advanced AI, while it could be Tool in the sense of not taking actions that impact the outside world, is likely to be Agent in the sense of optimizing within bounds internally.”
Out of the two implicit definitions of agent: “maximises UF” and “affects outside world (without explicit being told to)”, the second is the only one that is relevant to AI safety, and the one that is used by the actual AI community. IOW, trying to bring in the first definition just causes confusion.
After some more reading, particularly the Drexler CAIS report, I realize I was more confused than I thought about Tool vs Agent AI. I think I’ve resolved it, but I’d appreciate feedback. Would the below be correct?
“Most sophisticated software behaves like both a Tool and an Agent, at different times. Google Maps reports possible routes like a tool, but it searches for paths to maximize a utility function like an agent. DeepMind might ultimately select the move that maximizes its winning probability, but it follows some set rules in how it frames and conducts this search. Only relatively simple programs could be called purely Tool: Atari Pong genuinely runs no search function. Any advanced AI, while it could be Tool in the sense of not taking actions that impact the outside world, is likely to be Agent in the sense of optimizing within bounds internally.”
Out of the two implicit definitions of agent: “maximises UF” and “affects outside world (without explicit being told to)”, the second is the only one that is relevant to AI safety, and the one that is used by the actual AI community. IOW, trying to bring in the first definition just causes confusion.
Didn’t realize that, but it makes complete sense. Thanks.