After you’ve spent some time working in the framework of a decision theory where dynamic inconsistencies naturally Don’t Happen—not because there’s an extra clause forbidding them, but because the simple foundations just don’t give rise to them—then an intertemporal preference reversal starts looking like just another preference reversal.
… Roughly, self-modifying capability in a classical causal decision theorist doesn’t fix the problem that gives rise to the intertemporal preference reversals, it just makes one temporal self win out over all the others.
This is a genuine concern. Note that most instances of precommitment arise quite naturally due to reputational concerns: any agent which is complex enough to come up with the concept of reputation will make superficially irrational (“hawkish”) choices in order not to be pushed around in the future. Moreover, precommitment is only worthwhile if it can be accurately assessed by the counterparty: an agent will not want to “generally modify its future self … to do what its past self would have wished” unless it can gain a reputational advantage by doing so.
After you’ve spent some time working in the framework of a decision theory where dynamic inconsistencies naturally Don’t Happen—not because there’s an extra clause forbidding them, but because the simple foundations just don’t give rise to them—then an intertemporal preference reversal starts looking like just another preference reversal.
… Roughly, self-modifying capability in a classical causal decision theorist doesn’t fix the problem that gives rise to the intertemporal preference reversals, it just makes one temporal self win out over all the others.
This is a genuine concern. Note that most instances of precommitment arise quite naturally due to reputational concerns: any agent which is complex enough to come up with the concept of reputation will make superficially irrational (“hawkish”) choices in order not to be pushed around in the future. Moreover, precommitment is only worthwhile if it can be accurately assessed by the counterparty: an agent will not want to “generally modify its future self … to do what its past self would have wished” unless it can gain a reputational advantage by doing so.