I think that resolves most of our disagreement, and I’ll think a bit more about your current position. (Have to go to sleep now.) In the mean time, can you please make a correction to your post? As you can see, my argument isn’t “our wireheaded selves would probably prefer to be wireheaded” but rather “an FAI might tell us that we would prefer to be wireheaded if we knew what it felt like.” I guess you had in your mind the previous argument you heard from others, and conflated mine with theirs.
I think that resolves most of our disagreement, and I’ll think a bit more about your current position. (Have to go to sleep now.) In the mean time, can you please make a correction to your post? As you can see, my argument isn’t “our wireheaded selves would probably prefer to be wireheaded” but rather “an FAI might tell us that we would prefer to be wireheaded if we knew what it felt like.” I guess you had in your mind the previous argument you heard from others, and conflated mine with theirs.
Correction added.