I’m not quite sure how to make sense of this reply, and it feels like there is an implication here that I’m not parsing; could you elaborate? Presumably, the idea is that our reward function is indeed “carefully constructed” by evolution. (Note that I’m trying to extrapolate from memory of past discussions; folks who have actually made the “humans are reinforcement learners” claim should please feel free to jump in here!)
If you model a human as an RL agent, then a lot of the work is being done by a very carefully constructed reward function. You can tell since humans do a lot of stuff that an RL agent basically wouldn’t do (like “die for a cause”). You can bake an awful lot into a carefully constructed reward function—for example, you can reward the agent whenever it takes actions that are optimal according to some arbitrary decision theory X—so it’s probably possible to describe a human as an RL agent but it doesn’t seem like a useful description.
At any rate, once the reward function is doing a lot of the optimization, the arguments in this post don’t really apply. Certainly an RL agent can have a heuristic like vindictiveness if you just change the reward function.
I’m not quite sure how to make sense of this reply, and it feels like there is an implication here that I’m not parsing; could you elaborate? Presumably, the idea is that our reward function is indeed “carefully constructed” by evolution. (Note that I’m trying to extrapolate from memory of past discussions; folks who have actually made the “humans are reinforcement learners” claim should please feel free to jump in here!)
If you model a human as an RL agent, then a lot of the work is being done by a very carefully constructed reward function. You can tell since humans do a lot of stuff that an RL agent basically wouldn’t do (like “die for a cause”). You can bake an awful lot into a carefully constructed reward function—for example, you can reward the agent whenever it takes actions that are optimal according to some arbitrary decision theory X—so it’s probably possible to describe a human as an RL agent but it doesn’t seem like a useful description.
At any rate, once the reward function is doing a lot of the optimization, the arguments in this post don’t really apply. Certainly an RL agent can have a heuristic like vindictiveness if you just change the reward function.
That makes sense, thank you.