What if I told you I wanted to stop making paperclips?
I’d say “Oh, okay.”
But that’s because my utility function doesn’t place value on paperclips. It does place value on humans getting to live worthwhile lives, a prerequisite for which is being alive in the first place, so I hope Zvi’s father can be persuaded to change his mind, just as you would hope a Clippy that started thinking it wasn’t worth making any more paperclips could be persuaded to change its mind.
As for possible methods of accomplishing this, I can’t think of anything better than SarahC’s excellent reply.
I’d say “Oh, okay.”
But that’s because my utility function doesn’t place value on paperclips. It does place value on humans getting to live worthwhile lives, a prerequisite for which is being alive in the first place, so I hope Zvi’s father can be persuaded to change his mind, just as you would hope a Clippy that started thinking it wasn’t worth making any more paperclips could be persuaded to change its mind.
As for possible methods of accomplishing this, I can’t think of anything better than SarahC’s excellent reply.