The boundaries between present-day people and non-people can be sharper, by a fiat of many intervening class members being nonexistent, than the ideal categories. In other words, except for chimpanzees, cryonics patients, Terry Schiavo, and babies who are exactly 1 year and 2 months and 5 days old, there isn’t much that’s ambiguous between person and non-person.
More to the point, a CEV-based AI has a potentially different definition of ‘sentient being’ and ‘the class I am to extrapolate’. Theoretically you could be given the latter definition by pointing and not worry too much about boundary cases, and let it work out the former class by itself—if you were sure that the FAI would arrive at the correct answer without creating any sentients along the way!
The boundaries between present-day people and non-people can be sharper, by a fiat of many intervening class members being nonexistent, than the ideal categories.
Fair point.
More to the point, a CEV-based AI has a potentially different definition of ‘sentient being’ and ‘the class I am to extrapolate’. Theoretically you could be given the latter definition by pointing
Mm. Theoretically, yes, I suppose someone could point to every person, and I could be constructed so as to not generalize the extrapolated class beyond the particular targets I’ve been given.
I’m not sure I would endorse that, but I think that gets us into questions of what the extrapolated class ought to comprise in the first place, which is a much larger and mostly tangential discussion.
In other words, except for chimpanzees, cryonics patients, Terry Schiavo, and babies who are exactly 1 year and 2 months and 5 days old, there isn’t much that’s ambiguous between person and non-person.
Slightly offtopic, but doesn’t that assume personhood is binary? I’ve always assumed it was a sliding scale (I care far less about a dog compared to a human, but I care even less about a fly getting it’s wings pulled off. And even then, I care more than about a miniature clockwork fly.)
The boundaries between present-day people and non-people can be sharper, by a fiat of many intervening class members being nonexistent, than the ideal categories. In other words, except for chimpanzees, cryonics patients, Terry Schiavo, and babies who are exactly 1 year and 2 months and 5 days old, there isn’t much that’s ambiguous between person and non-person.
More to the point, a CEV-based AI has a potentially different definition of ‘sentient being’ and ‘the class I am to extrapolate’. Theoretically you could be given the latter definition by pointing and not worry too much about boundary cases, and let it work out the former class by itself—if you were sure that the FAI would arrive at the correct answer without creating any sentients along the way!
Fair point.
Mm. Theoretically, yes, I suppose someone could point to every person, and I could be constructed so as to not generalize the extrapolated class beyond the particular targets I’ve been given.
I’m not sure I would endorse that, but I think that gets us into questions of what the extrapolated class ought to comprise in the first place, which is a much larger and mostly tangential discussion.
So, fair enough… point taken.
Slightly offtopic, but doesn’t that assume personhood is binary? I’ve always assumed it was a sliding scale (I care far less about a dog compared to a human, but I care even less about a fly getting it’s wings pulled off. And even then, I care more than about a miniature clockwork fly.)