Yes. I picked the ethical formulation as a way to make clear that this isn’t just a terminological problem.
I like the framing in terms of expectation.
And I agree that this line of thought makes me skeptical about the computationalist theory of mind. The conventional formulations of computation seem to abstract away enough stuff about identity that you just can’t hang a theory of mind and future expectation on what’s left.
I think that arguments like this are a good reason to doubt computationalism. That means accepting that two systems performing the same computations can have different experiences, even though they behave in exactly the same way. But we already should have suspected this: it’s just like the inverted spectrum problem, where you and I both call the same flower “red,” but the subjective experience I have is what you would call “green” if you had it. We know that most computations even in our brains are not accompanied by conscious perceptual experience, so it shouldn’t be surprising if we can make a system that does whatever we want, but does it unconsciously.
Yes. I picked the ethical formulation as a way to make clear that this isn’t just a terminological problem.
I like the framing in terms of expectation.
And I agree that this line of thought makes me skeptical about the computationalist theory of mind. The conventional formulations of computation seem to abstract away enough stuff about identity that you just can’t hang a theory of mind and future expectation on what’s left.
I think that arguments like this are a good reason to doubt computationalism. That means accepting that two systems performing the same computations can have different experiences, even though they behave in exactly the same way. But we already should have suspected this: it’s just like the inverted spectrum problem, where you and I both call the same flower “red,” but the subjective experience I have is what you would call “green” if you had it. We know that most computations even in our brains are not accompanied by conscious perceptual experience, so it shouldn’t be surprising if we can make a system that does whatever we want, but does it unconsciously.