Point conceded: inconsistent preferences do not imply a practical exploitable attack vector (aka “money/paperclip pump”). However, it is common in game-theoretical discussions for e.g. intransitive preferences to not actually hurt agents that hold them, and yet the inconsistency is treated as if it opened the agent to paperclip pumping.
For example, in the Allais problem, people have intransitive preferences, and Editor:Eliezer_Yudkowsky has specified exactly how you would money-pump such a person. Yet it requires very contorted, atypical situations to actually perform the money pump.
Point conceded: inconsistent preferences do not imply a practical exploitable attack vector (aka “money/paperclip pump”). However, it is common in game-theoretical discussions for e.g. intransitive preferences to not actually hurt agents that hold them, and yet the inconsistency is treated as if it opened the agent to paperclip pumping.
For example, in the Allais problem, people have intransitive preferences, and Editor:Eliezer_Yudkowsky has specified exactly how you would money-pump such a person. Yet it requires very contorted, atypical situations to actually perform the money pump.