This thought experiment is unrealistic
many such cases
They’re not intended to be realistic, but as a way of disentangling different intuitions.
This perspective is captured here:
https://www.lesswrong.com/posts/s9hTXtAPn2ZEAWutr/please-don-t-fight-the-hypothetical
Where you shouldn’t try to fight the hypothetical’s assumptions, but instead actually answer the question.
Hypothetical question for you: what if there is a sequence of wrong mental turns that leads towards a psychological attractor state that’s bad, and accepting the hypothetical in the OP is one of these turns?
Then you shouldn’t accept the hypothetical.
I didn’t mean for this to be a claim that you must accept all hypotheticals.
many such cases
They’re not intended to be realistic, but as a way of disentangling different intuitions.
This perspective is captured here:
https://www.lesswrong.com/posts/s9hTXtAPn2ZEAWutr/please-don-t-fight-the-hypothetical
Where you shouldn’t try to fight the hypothetical’s assumptions, but instead actually answer the question.
Hypothetical question for you: what if there is a sequence of wrong mental turns that leads towards a psychological attractor state that’s bad, and accepting the hypothetical in the OP is one of these turns?
Then you shouldn’t accept the hypothetical.
I didn’t mean for this to be a claim that you must accept all hypotheticals.