Because, unlike the robot, the cognitive architectures producing the observed behavior (alleviating a pain) are likely to be similar to those producing the similar behavior in us (since evolution is likely to have reused the same cognitive architecture in us and in the fish), and we know that whatever cognitive architecture produces that behavior in us produces a pain quale. The worry was supposed to be that perhaps the underlying cognitive architecture is more like a reflex than like a conscious experience, but the way the experiment was set up precluded that, since it’s highly unlikely that a fish would have a reflex built in for this specific situation (unlike, say, the situation of pulling away from a hot object or a sharp object, which could be an unconscious reflex in other animals).
Because, unlike the robot, the cognitive architectures producing the observed behavior (alleviating a pain) are likely to be similar to those producing the similar behavior in us (since evolution is likely to have reused the same cognitive architecture in us and in the fish), and we know that whatever cognitive architecture produces that behavior in us produces a pain quale. The worry was supposed to be that perhaps the underlying cognitive architecture is more like a reflex than like a conscious experience, but the way the experiment was set up precluded that, since it’s highly unlikely that a fish would have a reflex built in for this specific situation (unlike, say, the situation of pulling away from a hot object or a sharp object, which could be an unconscious reflex in other animals).