Eliezer was using “sentient” practically as a synonym for “morally significant”. Everything he said about the hazards of creating sentient beings was about that. It’s true that in our current state, our feelings of morality come from empathic instincts, which may not stretch (without introspection) so far as to feel concern for a program which implements the algorithms of consciousness and cognition, even perhaps if it’s a human brain simulation. However, upon further consideration and reflection, we (or at least most of us, I think) find that a human brain simulation is morally significant, even though there is much that is not clear about the consequences. The same should be true of a consciousness that isn’t in fact a simulation of a human, but of course determining what is and what is not conscious is the hard part.
It would be a mistake to create a new species that deserves our moral consideration, even if at present we would not give it the moral consideration it deserves.
Tim:
Eliezer was using “sentient” practically as a synonym for “morally significant”. Everything he said about the hazards of creating sentient beings was about that. It’s true that in our current state, our feelings of morality come from empathic instincts, which may not stretch (without introspection) so far as to feel concern for a program which implements the algorithms of consciousness and cognition, even perhaps if it’s a human brain simulation. However, upon further consideration and reflection, we (or at least most of us, I think) find that a human brain simulation is morally significant, even though there is much that is not clear about the consequences. The same should be true of a consciousness that isn’t in fact a simulation of a human, but of course determining what is and what is not conscious is the hard part.
It would be a mistake to create a new species that deserves our moral consideration, even if at present we would not give it the moral consideration it deserves.