Say more about behaviors associated with “incomparability”?
Depending on the implementation details of the agent design, it may do some combination of:
Turning down your offer, path-dependently preferring whichever option is already in hand / whichever option is consistent with its history of past trades.
Noticing unresolved conflicts within its preference framework, possibly unresolveable without self-modifying into an agent that has different preferences from itself.
Halting and catching fire, folding under the weight of an impossible choice.
EDIT: The post also suggests an alternative (better) policy that agents with incomplete preferences may follow.
Say more about behaviors associated with “incomparability”?
Depending on the implementation details of the agent design, it may do some combination of:
Turning down your offer, path-dependently preferring whichever option is already in hand / whichever option is consistent with its history of past trades.
Noticing unresolved conflicts within its preference framework, possibly unresolveable without self-modifying into an agent that has different preferences from itself.
Halting and catching fire, folding under the weight of an impossible choice.
EDIT: The post also suggests an alternative (better) policy that agents with incomplete preferences may follow.