This depends entirely on context and specifics. How did I get such control (and what does “control” even mean, for something agentic)? How do I know it’s the first, and how far ahead of second is it? What can this agent do that my human collaborators or employees can’t?
In the sci-fi version, where it’s super-powerful and able to plan and execute, but only has goals that I somehow verbalize, I think Eliezer’s genie description (from https://www.lesswrong.com/posts/4ARaTpNX62uaL86j6/the-hidden-complexity-of-wishes) fits: ’There are three kinds of genies: Genies to whom you can safely say “I wish for you to do what I should wish for”; genies for which no wish is safe; and genies that aren’t very powerful or intelligent’. Which one is this?
A more interesting framing of a similar question is: for the people working to bring about agentic powerful AI, what goals are you trying to imbue into it’s agency?
Thanks for the comment. I agree that context and specifics are key. This is what I was trying to get at with “If you’d like to change or add to these assumptions for your answer, please spell out how.”
By “controlled,” I basically mean it does what I actually want it to do, filling in the unspecified blanks at least as well as a human would to follow as closely as it can to my true meaning/desire.
Thanks for your “more interesting framing” version. Part of the point of this post was to give AGI developers food for thought about what they might want to prioritize for their first AGI to do.
This depends entirely on context and specifics. How did I get such control (and what does “control” even mean, for something agentic)? How do I know it’s the first, and how far ahead of second is it? What can this agent do that my human collaborators or employees can’t?
In the sci-fi version, where it’s super-powerful and able to plan and execute, but only has goals that I somehow verbalize, I think Eliezer’s genie description (from https://www.lesswrong.com/posts/4ARaTpNX62uaL86j6/the-hidden-complexity-of-wishes) fits: ’There are three kinds of genies: Genies to whom you can safely say “I wish for you to do what I should wish for”; genies for which no wish is safe; and genies that aren’t very powerful or intelligent’. Which one is this?
A more interesting framing of a similar question is: for the people working to bring about agentic powerful AI, what goals are you trying to imbue into it’s agency?
Thanks for the comment. I agree that context and specifics are key. This is what I was trying to get at with “If you’d like to change or add to these assumptions for your answer, please spell out how.”
By “controlled,” I basically mean it does what I actually want it to do, filling in the unspecified blanks at least as well as a human would to follow as closely as it can to my true meaning/desire.
Thanks for your “more interesting framing” version. Part of the point of this post was to give AGI developers food for thought about what they might want to prioritize for their first AGI to do.