I think that arguments like this are a good reason to doubt computationalism. That means accepting that two systems performing the same computations can have different experiences, even though they behave in exactly the same way. But we already should have suspected this: it’s just like the inverted spectrum problem, where you and I both call the same flower “red,” but the subjective experience I have is what you would call “green” if you had it. We know that most computations even in our brains are not accompanied by conscious perceptual experience, so it shouldn’t be surprising if we can make a system that does whatever we want, but does it unconsciously.
I think that arguments like this are a good reason to doubt computationalism. That means accepting that two systems performing the same computations can have different experiences, even though they behave in exactly the same way. But we already should have suspected this: it’s just like the inverted spectrum problem, where you and I both call the same flower “red,” but the subjective experience I have is what you would call “green” if you had it. We know that most computations even in our brains are not accompanied by conscious perceptual experience, so it shouldn’t be surprising if we can make a system that does whatever we want, but does it unconsciously.