any possible set of behaviors can be construed as maximizing some utility function
(Edit: What do you mean? This calls to mind a basic introduction to what utility functions do, given below, but on second thought that’s probably not what the claim is about. I’ll leave the rest of the comment here, as it could be useful for someone.)
A utility function describes decisions between lotteries, which are mixtures of outcomes, or more generally events in a sample space. The setting assumes uncertainty, outcomes are only known to be within some event, not individually. So a situation where a decision can be made is a collection of events/lotteries, one of which gets to be chosen, the choice is the behavior assigned to this situation. This makes situations reuse parts of each other, they are not defined independently. As a result, it becomes possible to act incoherently, for example pick A from (A, B), pick B from (B, C) and pick C from (A, C). Only satisfying certain properties of collections of behaviors allows existence of a probability measure and a utility function such that agent’s choice among the collection of events in any situation coincides with picking the event that has the highest expected utility.
Put differently, the issue is that behavior described by a utility function is actually behavior in all possible and counterfactual situations, not in some specific situation. Existence of a utility function says something about which behaviors in different situations can coexist. Without a utility function, each situation could get an arbitrary response/behavior of its own, independently from the responses given for other situations. But requiring a utility function makes that impossible, some behaviors become incompatible with the other behaviors.
In the grandparent comment, I’m treating utility functions more loosely, but their role in constraining collections of behaviors assigned to different situations is the same.
(Edit: What do you mean? This calls to mind a basic introduction to what utility functions do, given below, but on second thought that’s probably not what the claim is about. I’ll leave the rest of the comment here, as it could be useful for someone.)
A utility function describes decisions between lotteries, which are mixtures of outcomes, or more generally events in a sample space. The setting assumes uncertainty, outcomes are only known to be within some event, not individually. So a situation where a decision can be made is a collection of events/lotteries, one of which gets to be chosen, the choice is the behavior assigned to this situation. This makes situations reuse parts of each other, they are not defined independently. As a result, it becomes possible to act incoherently, for example pick A from (A, B), pick B from (B, C) and pick C from (A, C). Only satisfying certain properties of collections of behaviors allows existence of a probability measure and a utility function such that agent’s choice among the collection of events in any situation coincides with picking the event that has the highest expected utility.
Put differently, the issue is that behavior described by a utility function is actually behavior in all possible and counterfactual situations, not in some specific situation. Existence of a utility function says something about which behaviors in different situations can coexist. Without a utility function, each situation could get an arbitrary response/behavior of its own, independently from the responses given for other situations. But requiring a utility function makes that impossible, some behaviors become incompatible with the other behaviors.
In the grandparent comment, I’m treating utility functions more loosely, but their role in constraining collections of behaviors assigned to different situations is the same.