There’s often not just one possible future self. First you choose which future self you wish to become, and then you update based on the credence of that self. Eliezer wants to become the future-Eliezer who knows the topic which the homework taught, so he updates accordingly. Before doing the homework, there’s a possible-future-Eliezer who blew off the class to watch cartoons and learned nothing. Eliezer has to choose which of those future-Eliezers he wants to update toward becoming, just as you have to choose which of your possible future selves you would prefer to be.
If you are about to be forced into the brainwashing machine, there’s a high probability that you will be brainwashed if you do nothing about it. But if you want to become the possible future self who does not desire to hold inaccurate beliefs indefinitely, you can update to that later future self’s rediscovery that the moon is made of rock wrapped in regolith.
Your new password is long and complicated. You confidently predict that, if you do nothing about it, you will have forgotten the password by this time next week. Since you desire to not forget your password, you choose to draw a picture that reminds you of what the password was, and type it out a few extra times. There’s a possible future self who has memorized the long and complex password, and you update yourself to more closely resemble that future self, perhaps by taking actions which they would have to have taken.
In the least convenient possible world, the brainwashing machine is very effective and you will be living the rest of your life underground with no access to any info about the moon.
The password won’t be needed after tomorrow, there is no point remembering it longer than that.
First you choose which future self you wish to become, and then you update based on the credence of that self.
I wish to become a future self that knows everything, wait I can’t just update my beliefs to match a hypothetical omniscient future version of myself, thus becoming omniscient.
If you want you future beliefs to be more accurate, update your beliefs to be more accurate. True advice, but not very useful.
I wish to become a future self that is confident they are living in a post ASI utopia. But deluding myself into thinking I am in a post ASI utopia despite strong evidence to the contrary isn’t a good idea.
There’s often not just one possible future self. First you choose which future self you wish to become, and then you update based on the credence of that self. Eliezer wants to become the future-Eliezer who knows the topic which the homework taught, so he updates accordingly. Before doing the homework, there’s a possible-future-Eliezer who blew off the class to watch cartoons and learned nothing. Eliezer has to choose which of those future-Eliezers he wants to update toward becoming, just as you have to choose which of your possible future selves you would prefer to be.
If you are about to be forced into the brainwashing machine, there’s a high probability that you will be brainwashed if you do nothing about it. But if you want to become the possible future self who does not desire to hold inaccurate beliefs indefinitely, you can update to that later future self’s rediscovery that the moon is made of rock wrapped in regolith.
Your new password is long and complicated. You confidently predict that, if you do nothing about it, you will have forgotten the password by this time next week. Since you desire to not forget your password, you choose to draw a picture that reminds you of what the password was, and type it out a few extra times. There’s a possible future self who has memorized the long and complex password, and you update yourself to more closely resemble that future self, perhaps by taking actions which they would have to have taken.
In the least convenient possible world, the brainwashing machine is very effective and you will be living the rest of your life underground with no access to any info about the moon.
The password won’t be needed after tomorrow, there is no point remembering it longer than that.
I wish to become a future self that knows everything, wait I can’t just update my beliefs to match a hypothetical omniscient future version of myself, thus becoming omniscient.
If you want you future beliefs to be more accurate, update your beliefs to be more accurate. True advice, but not very useful.
I wish to become a future self that is confident they are living in a post ASI utopia. But deluding myself into thinking I am in a post ASI utopia despite strong evidence to the contrary isn’t a good idea.