An entity with incomplete preferences can be inexploitable (= does not take sure losses) but it generically leaves sure gains on the table.
It seems like this is only the case if you apply the subagent vetocracy model. I agree that “an incomplete egregore/agent is like a ‘vetocracy’ of VNM subagents”, however, this is not the only valid model. There are other models of this that would not leave sure gains on the table.
The presence of a pre-order doesn’t inherently imply a composition of subagents with ordered preferences. An agent can have a pre-order of preferences due to reasons such as lack of information, indifference between choices, or bounds on computation—this does not necessitate the presence of subagents.
If we do not use a model based on composition of subagents with ordered preferences, in the case of “Atticus the Agent” it can be consistent to switch B → A + 1$ and A → B + 1$.
Perhaps I am misunderstanding the claim being made here though.
I think the model of “a composition of subagents with total orders on their preferences” is a descriptive model of inexploitable incomplete preferences, and not a mechanistic model. At least, that was how I interpreted “Why Subagents?”.
I read @johnswentworth as making the claim that such preferences could be modelled as a vetocracy of VNM rational agents, not as claiming that humans (or other objects of study) are mechanistically composed of discrete parts that are themselves VNM rational.
I’d be more interested/excited by a refutation on the grounds of: “incomplete inexploitable preferences are not necessarily adequately modelled as a vetocracy of parts with complete preferences”. VNM rationality and expected utility maximisation is mostly used as a descriptive rather than mechanistic tool anyway.
It seems like this is only the case if you apply the subagent vetocracy model. I agree that “an incomplete egregore/agent is like a ‘vetocracy’ of VNM subagents”, however, this is not the only valid model. There are other models of this that would not leave sure gains on the table.
Oh, do please share.
The presence of a pre-order doesn’t inherently imply a composition of subagents with ordered preferences. An agent can have a pre-order of preferences due to reasons such as lack of information, indifference between choices, or bounds on computation—this does not necessitate the presence of subagents.
If we do not use a model based on composition of subagents with ordered preferences, in the case of “Atticus the Agent” it can be consistent to switch B → A + 1$ and A → B + 1$.
Perhaps I am misunderstanding the claim being made here though.
I think the model of “a composition of subagents with total orders on their preferences” is a descriptive model of inexploitable incomplete preferences, and not a mechanistic model. At least, that was how I interpreted “Why Subagents?”.
I read @johnswentworth as making the claim that such preferences could be modelled as a vetocracy of VNM rational agents, not as claiming that humans (or other objects of study) are mechanistically composed of discrete parts that are themselves VNM rational.
I’d be more interested/excited by a refutation on the grounds of: “incomplete inexploitable preferences are not necessarily adequately modelled as a vetocracy of parts with complete preferences”. VNM rationality and expected utility maximisation is mostly used as a descriptive rather than mechanistic tool anyway.
I think you have misunderstood. In particular, you can still model agents that are incomplete because of e.g. bounded compute as vetocracies.
Oh I agree you can model any incomplete agents as vetocracies.
I am just pointing out that the argument:
You can model X using Y
Y implies Z
Does not imply:
Therefore Z for all X