You phrased your thought experiments in terms of good and bad lives. But what about the good and bad parts of a life? When you imagine a lot of good lives, you say that the value doesn’t add up. But what about the bad parts of those lives? Do you imagine that still accumulates and wipes out the value in the good parts?
One important caveat I didn’t specify is that I’m seeing this entire frame through the lens of “this is what we worry about after we’ve won. At the very least, once there’s been a major transition to ‘digital minds’, (where identical lives start to matter), and possibly, once we generally control as much of the lightcone as we were going to control in a stable fashion.
And the question is less formed around “how do we optimize individual lives”, and more formed around “in the limit, what do we do when it comes to what-sort-of-lives we create, and what sort of societies-that-allow-for-and-create-lives do we create. And possibly ’if we have the ability to intervene in other places where minds exist, either causally or through acausal trade, how do we want to influence that?”
2.
I’m happy with weird hacks that say ’don’t take any of this too seriously and if you’re considering either destroying the universe or doing weird goodharty things to solve the problem Repugnant Solution style, please stop, build yourself a Jupiter Brain, and think for a thousand years first (obviously spending the first year or so thinking about how to remain sane while being a Jupiter Brain for 1000 years). Meanwhile, try not to take drastic actions.
...
With that in mind, I think my answer to your question is “I think [very weakly] that it makes more sense to check whether a being endorses having lived”, rather than total experiences (with some caveats of I can imagine beings designed in a sadistic fashion to endorse having lived through torture and I might have complicated opinions about how to treat those being’s ability to endorse things)
An alternate hypothetical scheme might be, for a given life, to treat each moment as a “life” that ranges from birth-to-present-moment. Say a being is born, has a good early life, then experiences some hardship that makes them suicidal, then later thinks that hardship was worthwhile in some way. You might treat them as an 1000s of lives rather than one, and ask each one if they endorse having been born. You might add an extra step of, for the instance where they’re suicidal, communicate* to them that in the future they endorsed having lived through the hardship.
If they *still* endorse suicide, I’d probably weight that ‘life’ as a negative value in the scale. (i.e. if of 100 time slices of a person, 70 of those time slices endorse having lived and 30 don’t, even after simulating them with some time for meta reflection on the whole thing and thinking about the game theory of what they endorse, then that counts as 70 good things and 30 bad things on the cosmic scale)
You phrased your thought experiments in terms of good and bad lives. But what about the good and bad parts of a life? When you imagine a lot of good lives, you say that the value doesn’t add up. But what about the bad parts of those lives? Do you imagine that still accumulates and wipes out the value in the good parts?
First a couple background frame things:
1.
One important caveat I didn’t specify is that I’m seeing this entire frame through the lens of “this is what we worry about after we’ve won. At the very least, once there’s been a major transition to ‘digital minds’, (where identical lives start to matter), and possibly, once we generally control as much of the lightcone as we were going to control in a stable fashion.
And the question is less formed around “how do we optimize individual lives”, and more formed around “in the limit, what do we do when it comes to what-sort-of-lives we create, and what sort of societies-that-allow-for-and-create-lives do we create. And possibly ’if we have the ability to intervene in other places where minds exist, either causally or through acausal trade, how do we want to influence that?”
2.
I’m happy with weird hacks that say ’don’t take any of this too seriously and if you’re considering either destroying the universe or doing weird goodharty things to solve the problem Repugnant Solution style, please stop, build yourself a Jupiter Brain, and think for a thousand years first (obviously spending the first year or so thinking about how to remain sane while being a Jupiter Brain for 1000 years). Meanwhile, try not to take drastic actions.
...
With that in mind, I think my answer to your question is “I think [very weakly] that it makes more sense to check whether a being endorses having lived”, rather than total experiences (with some caveats of I can imagine beings designed in a sadistic fashion to endorse having lived through torture and I might have complicated opinions about how to treat those being’s ability to endorse things)
An alternate hypothetical scheme might be, for a given life, to treat each moment as a “life” that ranges from birth-to-present-moment. Say a being is born, has a good early life, then experiences some hardship that makes them suicidal, then later thinks that hardship was worthwhile in some way. You might treat them as an 1000s of lives rather than one, and ask each one if they endorse having been born. You might add an extra step of, for the instance where they’re suicidal, communicate* to them that in the future they endorsed having lived through the hardship.
If they *still* endorse suicide, I’d probably weight that ‘life’ as a negative value in the scale. (i.e. if of 100 time slices of a person, 70 of those time slices endorse having lived and 30 don’t, even after simulating them with some time for meta reflection on the whole thing and thinking about the game theory of what they endorse, then that counts as 70 good things and 30 bad things on the cosmic scale)