See this discussion. Physical theories of human identity must equate the world of appearances, which is the only world that we actually know about, with some part of a posited world of “physical entities”. Everything from the world of appearances is a quale, but an AI with a computational-materialist philosophy only “knows” various hypotheses about what the physical entities are. The most it could do is develop a concept like “the type of physical entity which causes a human to talk about appearances”, but it still won’t spontaneously attach the right significance to such concepts (e.g. to a concept of pain).
I have agreed elsewhere that it is—remotely! - possible that an appropriately guided AI could solve the hard problems of consciousness and ethics before humans did, e.g. by establishing a fantastically detailed causal model of human thought, and contemplating the deliberations of a philosophical sim-human. But when even the humans guiding the AI abandon their privileged epistemic access to phenomenological facts, and personally imitate the AI’s limitations by restricting themselves to computational epistemology, then the project is doomed.
See this discussion. Physical theories of human identity must equate the world of appearances, which is the only world that we actually know about, with some part of a posited world of “physical entities”. Everything from the world of appearances is a quale, but an AI with a computational-materialist philosophy only “knows” various hypotheses about what the physical entities are. The most it could do is develop a concept like “the type of physical entity which causes a human to talk about appearances”, but it still won’t spontaneously attach the right significance to such concepts (e.g. to a concept of pain).
I have agreed elsewhere that it is—remotely! - possible that an appropriately guided AI could solve the hard problems of consciousness and ethics before humans did, e.g. by establishing a fantastically detailed causal model of human thought, and contemplating the deliberations of a philosophical sim-human. But when even the humans guiding the AI abandon their privileged epistemic access to phenomenological facts, and personally imitate the AI’s limitations by restricting themselves to computational epistemology, then the project is doomed.