One possibility might be phrasing it as “Maximize preference satisfaction for everyone who exists and ever will exist, but not for everyone who could possibly exist..”
This captures the intuition that it is bad to create people who have low levels of preference satisfaction, even if they don’t exist yet and hence can’t object to being created, while preserving the belief that existing people have a right to not create new people whose existence would seriously interfere with their desires. It does this without implying anti-natalism. I admit that the phrasing is a little clunky and needs refinement, and I’m sure a clever enough UFAI could find some way to screw it up, but I think it’s a big step towards resolving the issues you point out.
EDIT: Another possibility that I thought of is setting “creating new worthwhile lives” and “improving already worthwhile lives” as two separate values that have diminishing returns relative to each other. This is still vulnerable to some forms of repugant-conclusion type arguments, but it totally eliminates what I think is the most repugnant aspect of the RC—the idea that a Malthusian society might be morally optimal.
One possibility might be phrasing it as “Maximize preference satisfaction for everyone who exists and ever will exist, but not for everyone who could possibly exist..”
This captures the intuition that it is bad to create people who have low levels of preference satisfaction, even if they don’t exist yet and hence can’t object to being created, while preserving the belief that existing people have a right to not create new people whose existence would seriously interfere with their desires. It does this without implying anti-natalism. I admit that the phrasing is a little clunky and needs refinement, and I’m sure a clever enough UFAI could find some way to screw it up, but I think it’s a big step towards resolving the issues you point out.
EDIT: Another possibility that I thought of is setting “creating new worthwhile lives” and “improving already worthwhile lives” as two separate values that have diminishing returns relative to each other. This is still vulnerable to some forms of repugant-conclusion type arguments, but it totally eliminates what I think is the most repugnant aspect of the RC—the idea that a Malthusian society might be morally optimal.