My impression is that a ton of work at MIRI (and some related research lines in other places) went into answering this question, and indeed, no one knows the answer very crisply right now and yup that’s alarming.
(Also, while I agree agents are “more mysterious than rocks or The Odyssey”, I’m actually confused why the circularity is particularly the problem here. Why doesn’t the Odyssey also run into the Abstraction for Whom problem?)
(Also, while I agree agents are “more mysterious than rocks or The Odyssey”, I’m actually confused why the circularity is particularly the problem here. Why doesn’t the Odyssey also run into the Abstraction for Whom problem?)
Oh, I think it does, actually. It’s just less immediate or central. Like, it’s easy for me to imagine putting a copy of The Odyssey on a computer. It’s damn near impossible for me to describe what putting “an agent” on my computer is, as opposed to some other kind of program. I was just trying to point at the center of the problem is all, and set aside the usual layers-of-abstraction “explanation” I’m used to hearing for this.
For low-bar definitions of “agent”, you’re probably running some already.
“In computer science, a software agent is a computer program that acts for a user or other program in a relationship of agency, which derives from the Latin agere (to do): an agreement to act on one’s behalf. Such “action on behalf of” implies the authority to decide which, if any, action is appropriate. Agents are colloquially known as bots, from robot.”
So, if you find agents weird and mysterious, you are probably using a different definition from Wikipedia’s article on software agents.
My impression is that a ton of work at MIRI (and some related research lines in other places) went into answering this question, and indeed, no one knows the answer very crisply right now and yup that’s alarming.
See John Wentworth’s post on Why Agent Foundations? An Overly Abstract Explanation, which discusses the need to find the True Name of agents.
(Also, while I agree agents are “more mysterious than rocks or The Odyssey”, I’m actually confused why the circularity is particularly the problem here. Why doesn’t the Odyssey also run into the Abstraction for Whom problem?)
Oh, I think it does, actually. It’s just less immediate or central. Like, it’s easy for me to imagine putting a copy of The Odyssey on a computer. It’s damn near impossible for me to describe what putting “an agent” on my computer is, as opposed to some other kind of program. I was just trying to point at the center of the problem is all, and set aside the usual layers-of-abstraction “explanation” I’m used to hearing for this.
For low-bar definitions of “agent”, you’re probably running some already.
“In computer science, a software agent is a computer program that acts for a user or other program in a relationship of agency, which derives from the Latin agere (to do): an agreement to act on one’s behalf. Such “action on behalf of” implies the authority to decide which, if any, action is appropriate. Agents are colloquially known as bots, from robot.”
So, if you find agents weird and mysterious, you are probably using a different definition from Wikipedia’s article on software agents.