It’s weird to think about what “respecting agency” means when the agent in question doesn’t currently exist and you are building it from scratch and you get to build it however you want. You can’t apply normal intuitions here.
For example, brainwashing a human such that they are only motivated to play tic-tac-toe is obviously not “respecting their agency”. We’re all on the same page for that situation.
But what if we build an AGI from scratch such that it is only motivated to play tic-tac-toe, and then we let the AGI do whatever it wants in the world (which happens to be playing tic-tac-toe)? Are we disrespecting its agency? If so, I don’t feel the force of the argument that this is bad. Who exactly are we harming here? Is it worse than not making this AGI in the first place?
Was evolution “disrespecting my agency” when I was born with a hunger drive and sex drive and status drive etc.? If not, why would it be any different to make an AGI with (only) a tic-tac-toe drive? Or if yes, well, we face the problem that we need to put some drives into our AGI or else there’s no “agent” at all, just a body that takes random actions, or doesn’t do anything at all.
I never talked about respect in my points for a reason. This isn’t about respect. It’s about how it is not an agent if it doesn’t do anything in an attempt to make the world more to its liking. If it does nothing, or does things randomly (without regard to making things better), that is hardly agentic. If I don’t care at all about colors, then picking between a red shirt, and an otherwise identical blue shirt is not an agentic choice, merely a necessary choice (and I will not likely have even bothered thinking about it as a choice involving color.). Identically, if I just have to always wear a specific color, I am not being an agent by wearing that color. There are obviously degrees of agency, too, but the article is genuinely assuming that beings that do basically nothing are still agents.
It’s weird to think about what “respecting agency” means when the agent in question doesn’t currently exist and you are building it from scratch and you get to build it however you want. You can’t apply normal intuitions here.
For example, brainwashing a human such that they are only motivated to play tic-tac-toe is obviously not “respecting their agency”. We’re all on the same page for that situation.
But what if we build an AGI from scratch such that it is only motivated to play tic-tac-toe, and then we let the AGI do whatever it wants in the world (which happens to be playing tic-tac-toe)? Are we disrespecting its agency? If so, I don’t feel the force of the argument that this is bad. Who exactly are we harming here? Is it worse than not making this AGI in the first place?
Was evolution “disrespecting my agency” when I was born with a hunger drive and sex drive and status drive etc.? If not, why would it be any different to make an AGI with (only) a tic-tac-toe drive? Or if yes, well, we face the problem that we need to put some drives into our AGI or else there’s no “agent” at all, just a body that takes random actions, or doesn’t do anything at all.
I never talked about respect in my points for a reason. This isn’t about respect. It’s about how it is not an agent if it doesn’t do anything in an attempt to make the world more to its liking. If it does nothing, or does things randomly (without regard to making things better), that is hardly agentic. If I don’t care at all about colors, then picking between a red shirt, and an otherwise identical blue shirt is not an agentic choice, merely a necessary choice (and I will not likely have even bothered thinking about it as a choice involving color.). Identically, if I just have to always wear a specific color, I am not being an agent by wearing that color. There are obviously degrees of agency, too, but the article is genuinely assuming that beings that do basically nothing are still agents.