The bearing this has on applied rationality is that this problem serves as a least convenient possible world
When the least convenient possible world is also the most impossible possible world, I find the exercise less than useful. It’s like Pascal’s Mugging. Sure, there can be things you’re better off not knowing, but the thing to do is to level up your ability to handle it. The fact that however powerful you imagine youself, you can imagine a more powerful Omega is like asking whether God can make a rock so heavy he can’t lift it.
When the least convenient possible world is also the most impossible possible world, I find the exercise less than useful. It’s like Pascal’s Mugging. Sure, there can be things you’re better off not knowing, but the thing to do is to level up your ability to handle it.
Leveling up is great, but I’m still not going to try to beat up an entire street-gang just to steal their bling. I don’t have that level of combat prowess right now even though it is entirely possible to level up enough for that kind of activity to be possible and safe. It so happens that neither I nor any non-fictional human is at that level or likely to be soon. In the same way there is a huge space of possible agent that would be able to calculate true information that it would be detrimental for me to have. For most humans just another particularly manipulative human would be enough and for all the rest any old superintellgence would do.
The fact that however powerful you imagine youself, you can imagine a more powerful Omega is like asking whether God can make a rock so heavy he can’t lift it.
No, this is a cop-out. Humans do encounter situations where they encounter agents more powerful than themselves, including agents that are more intelligent and able to exploit human weaknesses. Just imagining yourself to be more powerful and more able to “handle the truth” isn’t especially useful and trying to dismiss all such scenarios as like God combatting his own omnipotence would be irresponsible.
When the least convenient possible world is also the most impossible possible world, I find the exercise less than useful. It’s like Pascal’s Mugging. Sure, there can be things you’re better off not knowing, but the thing to do is to level up your ability to handle it. The fact that however powerful you imagine youself, you can imagine a more powerful Omega is like asking whether God can make a rock so heavy he can’t lift it.
Leveling up is great, but I’m still not going to try to beat up an entire street-gang just to steal their bling. I don’t have that level of combat prowess right now even though it is entirely possible to level up enough for that kind of activity to be possible and safe. It so happens that neither I nor any non-fictional human is at that level or likely to be soon. In the same way there is a huge space of possible agent that would be able to calculate true information that it would be detrimental for me to have. For most humans just another particularly manipulative human would be enough and for all the rest any old superintellgence would do.
No, this is a cop-out. Humans do encounter situations where they encounter agents more powerful than themselves, including agents that are more intelligent and able to exploit human weaknesses. Just imagining yourself to be more powerful and more able to “handle the truth” isn’t especially useful and trying to dismiss all such scenarios as like God combatting his own omnipotence would be irresponsible.
Omega isn’t showing up right now.
No non-fictional Omega is at that level either.
Then it would seem you need to delegate your decision theoretic considerations to those better suited to abstract analysis.