Also: I don’t understand how one is supposed to get from “trained agents have a variety of contextual influences on decision making” to “trained agents are not expected utility maximizers”, without somehow rebutting the arguments people make for why utility maximizers are good—and once we refute these arguments, we don’t need talk of “shards” to refute them extra hard. Like, you can have different influences on your behaviour at different times that all add up to coherence. For example, one obvious influence that would plausibly be reinforced is “think about coherence so you don’t randomly give up resources”. Maybe this is supposed to make more sense if we use “expected utility maximizer” in a way that excludes “thing that is almost expected-utility-optimal” or “thing that switches between different regimes of expected utility maximization” but that strikes me as silly.
Also: I don’t understand how one is supposed to get from “trained agents have a variety of contextual influences on decision making” to “trained agents are not expected utility maximizers”, without somehow rebutting the arguments people make for why utility maximizers are good—and once we refute these arguments, we don’t need talk of “shards” to refute them extra hard. Like, you can have different influences on your behaviour at different times that all add up to coherence. For example, one obvious influence that would plausibly be reinforced is “think about coherence so you don’t randomly give up resources”. Maybe this is supposed to make more sense if we use “expected utility maximizer” in a way that excludes “thing that is almost expected-utility-optimal” or “thing that switches between different regimes of expected utility maximization” but that strikes me as silly.