Yes I agree it feels fishy. The problem with maximizing rubes is that the dilemmas might get lost in the detail of preventing rube hacking. Perhaps agents can “paint” existing money their own color, and money can only be painted once, and agents want to paint as much money as possible. Then the details remain in the env
Or something simpler would be that the agent’s money counter is in the environment but unmodifiable except by getting tokens, and the agent’s goal is to maximize this quantity. Feels kind of fake maybe because money gives the agent no power or intelligence, but it’s a valid object-in-the-world to have a preference over the state of.
Yet another option is to have the agent maximize energy tokens (which actions consume)
Yes I agree it feels fishy. The problem with maximizing rubes is that the dilemmas might get lost in the detail of preventing rube hacking. Perhaps agents can “paint” existing money their own color, and money can only be painted once, and agents want to paint as much money as possible. Then the details remain in the env
Or something simpler would be that the agent’s money counter is in the environment but unmodifiable except by getting tokens, and the agent’s goal is to maximize this quantity. Feels kind of fake maybe because money gives the agent no power or intelligence, but it’s a valid object-in-the-world to have a preference over the state of.
Yet another option is to have the agent maximize energy tokens (which actions consume)