I have tripped over what the semantics of utility functions means. To me it is largely a lossless conversion of the preferences. I am also looking from the “outside” rather than the inside.
You can condition on internal state such that for example F(on the street & angry) can return something else than F(on the street & sad)
Suppose you have a situation that can at separate moments be modeled as separate functions, ie max_paperclips(x) for t_1 and max_horsehoes(x) for t_2 then you can form a function of ironsmith(t,x) that at once represents both.
Now suppose that some actions the agent can choose in max_paperclips “scrambles the insides” and max_horseshoes ceases to be representative for t_2 for some outcomes. Instead max_goldbars becomes representive. You can still form a function metalsmith(t,x,history) with history containing what the agent has done.
One can argue about what kind of properties the functions have but the mere existence of a function is hard to circumvent so atleast the option of using such terminology is quite sturdy.
Maybe you mean that a claim like “agent utility functions have time translation symmetry” is false, ie the behaviour is not static over time.
Maybe you mean that a claim like “agent utility functions are functions of their perception of the outside world only” is false? That agents are allowed to have memory.
Failing to be a function is really hard as the concept “just” translates preferences into another formulation.
If you allowed utility functions to condition on the entire history (observation, actions and any changes induced in the agent by the environment), you could describe agent preferences, but at that point you lose all the nice properties of EU maximisation.
You’re no longer prescribing how a rational agent should behave and the utility function no longer constrains agent behaviour (any action is EU maximisation). Whatever action the agent takes becomes the action with the highest utility.
Utility functions that condition on an agent’s history are IMO not useful for theories of normative decision making or rationality. They become a purely descriptive artifact.
If anyone can point me to what kinds of preferences are supposed to be outlawed by this “insist on being a utility function” I would benefit from that.
They say that you are allowed to define utility functions however you want, but that doing so broadly enough can mean that “X is behaving according to a utility function” is no longer anticipation-constraining, so you can’t infer anything new about X from it.
I have tripped over what the semantics of utility functions means. To me it is largely a lossless conversion of the preferences. I am also looking from the “outside” rather than the inside.
You can condition on internal state such that for example F(on the street & angry) can return something else than F(on the street & sad)
Suppose you have a situation that can at separate moments be modeled as separate functions, ie max_paperclips(x) for t_1 and max_horsehoes(x) for t_2 then you can form a function of ironsmith(t,x) that at once represents both.
Now suppose that some actions the agent can choose in max_paperclips “scrambles the insides” and max_horseshoes ceases to be representative for t_2 for some outcomes. Instead max_goldbars becomes representive. You can still form a function metalsmith(t,x,history) with history containing what the agent has done.
One can argue about what kind of properties the functions have but the mere existence of a function is hard to circumvent so atleast the option of using such terminology is quite sturdy.
Maybe you mean that a claim like “agent utility functions have time translation symmetry” is false, ie the behaviour is not static over time.
Maybe you mean that a claim like “agent utility functions are functions of their perception of the outside world only” is false? That agents are allowed to have memory.
Failing to be a function is really hard as the concept “just” translates preferences into another formulation.
If you allowed utility functions to condition on the entire history (observation, actions and any changes induced in the agent by the environment), you could describe agent preferences, but at that point you lose all the nice properties of EU maximisation.
You’re no longer prescribing how a rational agent should behave and the utility function no longer constrains agent behaviour (any action is EU maximisation). Whatever action the agent takes becomes the action with the highest utility.
Utility functions that condition on an agent’s history are IMO not useful for theories of normative decision making or rationality. They become a purely descriptive artifact.
If anyone can point me to what kinds of preferences are supposed to be outlawed by this “insist on being a utility function” I would benefit from that.
See: Why Subagents? for a treatment of how stateless utility functions fail to capture inexploitable path dependent preferences.
This and this are decent discussions.
So it seems i have understood correctly and both of those say that nothing is outlawed (and that incoherence is broken as a concept).
They say that you are allowed to define utility functions however you want, but that doing so broadly enough can mean that “X is behaving according to a utility function” is no longer anticipation-constraining, so you can’t infer anything new about X from it.