Excellent question, Gordon! I defined tool AI specifically as controllable, so AI without a quantitative guarantee that it’s controllable (or “safe”, as you write) wouldn’t meet the safety standards and its release would be prohibited. I think it’s crucial that, just as for aviation and pharma, the onus is on the companies rather than the regulators to demonstrate that products meet the safety standards. For controllable tools with great potential for harm (say plastic explosives), we already have regulatory approaches for limiting who can use them and how. Analogously, there’s discussion at the UNGA this week about creating a treaty on lethal autonomous weapons, which I support.
I defined tool AI specifically as controllable, so AI without a quantitative guarantee that it’s controllable (or “safe”, as you write) wouldn’t meet the safety standards and its release would be prohibited.
If your stated definition is really all you mean by tool AI, then you’ve defined tool AI in a very nonstandard way that will confuse your readers.
When most people hear “tool AI”, I expect them to think of AI like hammers: tools they can use to help them achieve a goal, but aren’t agentic and won’t do anything on their own they weren’t directly asked to do.
You seem to have adopted a definition of “tool AI” that actually means “controllable and goal-achieving AI”, but give no consideration to agency, so I can only conclude from your writing that you would mean for AI agents to be included as tools, even if they operated independently, so long as they could be controlled in some sense (what sense control takes exactly you never specify). This is not what I expect most people to expect someone to mean by a “tool”.
Again, I like all the reasoning about entente, but this use of the word “tool AI” is confusing, maybe even deceptive (I assume that was not the intent!). It also leaves me felling like your “solution” of tool AI is nothing other than a rebrand of what we’ve already been talking about in the field variously as safe, aligned, or controllable AI, which I guess is fine, but “tool AI” is a confusing name for that. This also further downgrades my opinion of the solution section, since as best I can tell it’s just saying “build AI safely” without enough details to be actionable.
Even if tool AI is controllable, tool AI can be used to assist in building non-tool AI. A benign superassistant is one query away from outputting world-ending code.
Right, Tamsin: so reasonable safety standards would presumably ban fully unrestricted superassistants too, but allow more limited assistants that could still be incredibly helpful. I’m curious what AI safety standards you’d propose – it’s not a hypothetical question, since many politicians would like to know.
Excellent question, Gordon! I defined tool AI specifically as controllable, so AI without a quantitative guarantee that it’s controllable (or “safe”, as you write) wouldn’t meet the safety standards and its release would be prohibited. I think it’s crucial that, just as for aviation and pharma, the onus is on the companies rather than the regulators to demonstrate that products meet the safety standards. For controllable tools with great potential for harm (say plastic explosives), we already have regulatory approaches for limiting who can use them and how. Analogously, there’s discussion at the UNGA this week about creating a treaty on lethal autonomous weapons, which I support.
If your stated definition is really all you mean by tool AI, then you’ve defined tool AI in a very nonstandard way that will confuse your readers.
When most people hear “tool AI”, I expect them to think of AI like hammers: tools they can use to help them achieve a goal, but aren’t agentic and won’t do anything on their own they weren’t directly asked to do.
You seem to have adopted a definition of “tool AI” that actually means “controllable and goal-achieving AI”, but give no consideration to agency, so I can only conclude from your writing that you would mean for AI agents to be included as tools, even if they operated independently, so long as they could be controlled in some sense (what sense control takes exactly you never specify). This is not what I expect most people to expect someone to mean by a “tool”.
Again, I like all the reasoning about entente, but this use of the word “tool AI” is confusing, maybe even deceptive (I assume that was not the intent!). It also leaves me felling like your “solution” of tool AI is nothing other than a rebrand of what we’ve already been talking about in the field variously as safe, aligned, or controllable AI, which I guess is fine, but “tool AI” is a confusing name for that. This also further downgrades my opinion of the solution section, since as best I can tell it’s just saying “build AI safely” without enough details to be actionable.
Even if tool AI is controllable, tool AI can be used to assist in building non-tool AI. A benign superassistant is one query away from outputting world-ending code.
Right, Tamsin: so reasonable safety standards would presumably ban fully unrestricted superassistants too, but allow more limited assistants that could still be incredibly helpful. I’m curious what AI safety standards you’d propose – it’s not a hypothetical question, since many politicians would like to know.