It occurs to me that maybe we could regard the agent as consistently reasoning, “If I choose of my own free will to output 2, that thereby causes Peano Arithmetic to be inconsistent, causing me to get 0 points.”
I mostly don’t buy this, but it slightly defends the legitness of the counterfactual.
Nice! Yes, I encourage you to develop this into a post.
I can’t see the grandparent, so posting here:
It occurs to me that maybe we could regard the agent as consistently reasoning, “If I choose of my own free will to output 2, that thereby causes Peano Arithmetic to be inconsistent, causing me to get 0 points.”
I mostly don’t buy this, but it slightly defends the legitness of the counterfactual.