I swear I’ll give you a PhD if you write the thesis. On fancy paper and everything.
Would timeless decision theory handle negotiation with your future self?
For example if a timeless decision agent likes paperclips today but you knows it is going to be modified to like apples tomorrow, (and not care a bit about paperclips,) will it abstain from destroying the apple orchard, and its future self abstain from destroying the paperclips in exchange?
And is negotiation the right way to think about reconciling the difference between what I now want and what a predicted smarter, grown up, more knowledgeable version of me would want? or am I going the wrong way?
to talk about turning a paperclip maximizer into an apple maximizer is needlessly confusing. Better to talk about destroying a paperclip maximizer and creating an apple maximizer. And yes, timeless decision theory should allow these two agents to negotiate, though it gets confusing fast.
In the paperclip->apple scenario, in the sense that it retains the memory and inherits the assets of the original, and everything else that keeps you ‘you’ when you start wanting something different.
I swear I’ll give you a PhD if you write the thesis. On fancy paper and everything.
Would timeless decision theory handle negotiation with your future self? For example if a timeless decision agent likes paperclips today but you knows it is going to be modified to like apples tomorrow, (and not care a bit about paperclips,) will it abstain from destroying the apple orchard, and its future self abstain from destroying the paperclips in exchange?
And is negotiation the right way to think about reconciling the difference between what I now want and what a predicted smarter, grown up, more knowledgeable version of me would want? or am I going the wrong way?
to talk about turning a paperclip maximizer into an apple maximizer is needlessly confusing. Better to talk about destroying a paperclip maximizer and creating an apple maximizer. And yes, timeless decision theory should allow these two agents to negotiate, though it gets confusing fast.
In what sense is that a future self?
In the paperclip->apple scenario, in the sense that it retains the memory and inherits the assets of the original, and everything else that keeps you ‘you’ when you start wanting something different.
In the simulation scenario, I’m not sure.