Hm. If there is a strong causal relationship between knowing truths and utility, then it is conceivable that this is a trick: the truth, while optimized for disutility, might still present me with a net gain over the falsehood and the utility. But honestly, I am not sure I buy that: you can get utility from a false belief, if that belief happens to steer you in such a way that it adds utility. You can’t normally count on that, but this is Omega we are talking about.
The ‘other related falsehoods and rationalizing’ part has me worried. The falsehood might net me utility in and of itself, but I poke and prod ideas. If my brain becomes set up so that I will rationalize this thought, altering my other thought-patterns to match as I investigate… that could be very bad for me in the long run. Really bad.
And I’ve picked inconvenient truth over beneficial falsehood before on those grounds, so I probably would pick the inconvenient truth. Maybe, if Omega guaranteed that the process of rationalizing the falsehood would still have me with more utility- over the entire span of my life- than I would earn normally after you factor in the disutility of the inconvenient truth then I would take the false-box. But on the problem as it stands, I’d take the truth-box.
That’s why the problem specified ‘long-term’ utility. Omega is essentially saying ‘I have here a lie that will improve your life as much as any lie possibly can, and a truth that will ruin your life as badly as any truth can; which would you prefer to believe?’
Yes, believing a lie does imply that your map has gotten worse, and rationalizing your belief in the lie (which we’re all prone to do to things we believe) will make it worse. Omega has specified that this lie has optimal utility among all lies that you, personally, might believe; being Omega, it is as correct in saying this as it is possible to be.
On the other hand, the box containing the least optimal truth is a very scary box. Presume first that you are particularly strong emotionally and psychologically; there is no fact that will directly drive you to suicide. Even so, there are probably facts out there that will, if comprehended and internalized, corrupt your utility function, leading you to work directly against all you currently believe in. There’s probably something even worse than that out there in the space of all possible facts, but the test is rated to your utility function when Omega first encountered you, so ‘you change your ethical beliefs, and proceed to spend your life working to spread disutility, as you formerly defined it’ is on the list of possibilities.
Interesting idea. That would imply that there is a fact out there that, once known, would change my ethical beliefs, which I take to be a large part of my utility function, AND would do so in such a way that afterward, I would assent to acting on the new utility function.
But one of the things that Me(now) values is updating my beliefs based on information. If there is a fact that shows that my utility function is misconstrued, I want to know it. I don’t expect such a fact to surface, but I don’t have a problem imagining such a fact existing. I’ve actually lost things that Me(past) valued highly on the basis of this, so I have some evidence that I would rather update my knowledge than maintain my current utility function. Even if that knowledge causes me to update my utility function so as not to prefer knowledge over keeping my utility function.
So I think I might still pick the truth. A more precise account for how much utility is lost or gained in each scenario might convince me otherwise, but I am still not sure that I am better off letting my map get corrupted as opposed to letting my values get corrupted, and I tend to pick truth over utility. (Which, in this scenario, might be suboptimal, but I am not sure it is.)
Hm. If there is a strong causal relationship between knowing truths and utility, then it is conceivable that this is a trick: the truth, while optimized for disutility, might still present me with a net gain over the falsehood and the utility. But honestly, I am not sure I buy that: you can get utility from a false belief, if that belief happens to steer you in such a way that it adds utility. You can’t normally count on that, but this is Omega we are talking about.
The ‘other related falsehoods and rationalizing’ part has me worried. The falsehood might net me utility in and of itself, but I poke and prod ideas. If my brain becomes set up so that I will rationalize this thought, altering my other thought-patterns to match as I investigate… that could be very bad for me in the long run. Really bad.
And I’ve picked inconvenient truth over beneficial falsehood before on those grounds, so I probably would pick the inconvenient truth. Maybe, if Omega guaranteed that the process of rationalizing the falsehood would still have me with more utility- over the entire span of my life- than I would earn normally after you factor in the disutility of the inconvenient truth then I would take the false-box. But on the problem as it stands, I’d take the truth-box.
That’s why the problem specified ‘long-term’ utility. Omega is essentially saying ‘I have here a lie that will improve your life as much as any lie possibly can, and a truth that will ruin your life as badly as any truth can; which would you prefer to believe?’
Yes, believing a lie does imply that your map has gotten worse, and rationalizing your belief in the lie (which we’re all prone to do to things we believe) will make it worse. Omega has specified that this lie has optimal utility among all lies that you, personally, might believe; being Omega, it is as correct in saying this as it is possible to be.
On the other hand, the box containing the least optimal truth is a very scary box. Presume first that you are particularly strong emotionally and psychologically; there is no fact that will directly drive you to suicide. Even so, there are probably facts out there that will, if comprehended and internalized, corrupt your utility function, leading you to work directly against all you currently believe in. There’s probably something even worse than that out there in the space of all possible facts, but the test is rated to your utility function when Omega first encountered you, so ‘you change your ethical beliefs, and proceed to spend your life working to spread disutility, as you formerly defined it’ is on the list of possibilities.
Interesting idea. That would imply that there is a fact out there that, once known, would change my ethical beliefs, which I take to be a large part of my utility function, AND would do so in such a way that afterward, I would assent to acting on the new utility function.
But one of the things that Me(now) values is updating my beliefs based on information. If there is a fact that shows that my utility function is misconstrued, I want to know it. I don’t expect such a fact to surface, but I don’t have a problem imagining such a fact existing. I’ve actually lost things that Me(past) valued highly on the basis of this, so I have some evidence that I would rather update my knowledge than maintain my current utility function. Even if that knowledge causes me to update my utility function so as not to prefer knowledge over keeping my utility function.
So I think I might still pick the truth. A more precise account for how much utility is lost or gained in each scenario might convince me otherwise, but I am still not sure that I am better off letting my map get corrupted as opposed to letting my values get corrupted, and I tend to pick truth over utility. (Which, in this scenario, might be suboptimal, but I am not sure it is.)