Does this assume there is some symmetry between the unimaginably bad outcomes and the unimaginably good outcomes?
It seems very clear to me that the worst outcomes are just so much more negative than the best outcomes are positive. I think that is just a fundamental aspect of how experience works.
even if that’s true, the odds difference more than makes up for it
The odds of a lot of people being tortured for eternity seems really small. The threat in a conflict with a compassionate AI is the only scenario I can think of where an AGI would do that. How likely is that? One in a million? A billion? And even in that case, is it going to really do it to a large number of people for a very long time? (That would imply that the threat failed, AND it won the conflict anyway, AND it’s going to follow through on the threat even though it no longer matters (but this isn’t probably important for overall odds so let’s not get hung up on this. The point is it’s a very specific scenario with low total odds).
The ratio between how good the best experiences are and how bad the worst pain is are maybe ten or a hundred times. Even people who’ve reported very bad pain that makes them want to die have been able to endure it for a long time. Similarly with the worst depressions.
So if we compare one in a million time one hundred (the worst estimates), we get one in ten thousand compated to maybe 50% of very very good long term outcomes. Expected pleasure is five thousand times (!) larger than expected suffering.
This is roughly a product of the fact that intelligent beings tend to want pleasure for themselves and each other. We’re trying to make aligned AGI. We’re not sure to succeed, but screwing it up so badly that we all get tortured is really unlikely. The few really bad sadists in the world aren’t going to get much say at all. So the odds are on are side, even though success is far from certain. Failure is much more likely to result in oblivion than torture. A good future is a “broad attractor” and a bad future is not.
it doesn’t need to stay that way.
That is a fundamental aspect of how experience works now. That’s also a result of evolution wiring us to pay more attention to bad things than good things.
That doesn’t need to stay how experience works. If we get the really good outcome, we get to re-wire our brains however we like. We could potentially be in a state of bliss while still thinking and doing stuff.
I appreciate the thoughtful response and that you seem to take the ideas seriously.
That is a fundamental aspect of how experience works now. That’s also a result of evolution wiring us to pay more attention to bad things than good things.
I do think it’s a fundamental aspect of how experience works, independently of how our brains are disposed to thinking about it, however I definitely think it’s possible to prophylactically shield or consciousness against the depths of suffering by modifying the substrate. I can’t tell whether we’re disagreeing or not.
I don’t know exactly how to phrase It, but I think a fundamental aspect of the universe is that as suffering increases in magnitude, it becomes less and less clear that there is (or can be) a commensurate value on the positive side which can negate it(trade off against it, even things out). I don’t think it’s true of the reverse.
Are you making the claim that this view is a faulty conclusion owing to the contingent disposition of my human brain?
Or are you making the claim that the disposition of my human brain can be modified so as to prevent exposure to the depths of suffering?
This is getting more complex, and I’m running out of time. So I’ll be really brief here and ask for clarification:
I don’t understand why you think suffering is primary outside of particular brain/mind wiring. I hope I’m misunderstanding you. That seems wildly unlikely to me, and like a very negative view of the world.
So, clarify that?
Your intuition that no amount of pleasure might make up for suffering is the view of negative utilitarians. I’ve spent some time engaging with that worldview and the people who hold it. I think it’s deeply, fundamentally mistaken. It appears to be held by people who have suffered much more than they’ve enjoyed life. Their logic doesn’t hold up to me. If you think an entity disliking its experience (life) is worth avoiding, it seems like the simple inverse (enjoying life, pleasure) is logically worth seeking. The two cancel in decision-making terms.
So yes, I do think suffering seems primary to you based on your own intuitions and your own (very common) human anxiety, and the cold logic doesn’t follow that inution.
Yes, I’m definitely saying that your brain can be modified so that you experience more pleasure than suffering. To me it seems that thinking otherwise is to believe that your brain isn’t the whole of your experience. It is substance dualism, which has very little support in terms of either good arguments or good proponents. We are our brains, or rather the pattern within them. Change that pattern and we change our experience. This has been demonstrated a million times with brain injuries, drugs, and other brain changes. If dualism is true, the world is a massive conspiracy to make us think otherwise. If that’s the case, none of this matters, so we should assume and act as though materialism is true and we are our brains. If that’s the case, we can modify our experience as we like, given sufficient technology. AGI will very likely supply sufficient technology.
I don’t understand why you think suffering is primary outside of particular brain/mind wiring. I hope I’m misunderstanding you. That seems wildly unlikely to me, and like a very negative view of the world.
Basically I think within the space of all possible varieties and extents of conscious experience, suffering starts to become less and less Commensurable with positive experience the further you go towards the extremes.
If option (A) is to experience the worst possible suffering for 100 years, prior to experiencing the greatest possible pleasure for N number of years, and option (B) is non existence, I would choose option (B), regardless of the value of N.
It appears to be held by people who have suffered much more than they’ve enjoyed life.
Should this count as evidence against their views? It seems clear to me that if you’re trying to understand the nature of qualitative states, first hand experience with extreme states is an asset.
I have personally experienced prolonged states of consciousness which were far worse than non-existence. Should that not play a part in informing my views? Currently I’m very happy, I fear death, I’ve experienced extraordinary prolonged pleasure states. Would you suggest I’m just not acquainted with levels of wellbeing which would cause me to meaningfully revaluate my view?
I think there’s also a sort of meta issue where people with influence are systematically less acquainted with direct experience of the extremes of suffering. Meaning that discourse and decision making will tend to systematically underweight experiences of suffering as a direct data source.
I’d also choose to not exist over the worst suffering for a hundred years—IF I was in my current brain-state. I’d be so insane as to be not-me after just a few minutes or hours if my synapses worked normally and my brain tried to adapt to that state. If I were forced to retain sanity and my character, it would be a harder choice if N got to be more than a hundred times longer.
Regardless, this intuition is just that. It doesn’t show that there’s something fundamentally more important about suffering than pleasure. Just that we’re better at imagining strong suffering than strong pleasure. Which is natural given the evolutionary incentives to focus us on pain.
I definitely didn’t mean to dismiss negative utilitarianism because some of the individuals who believe it seem damaged. I’m skeptical of it because it makes no sense to me, and discussions with NU people don’t help. The most rational among them slide back to negatively-balanced utilitarianism when they’re pressed on details—the FAQ I was pointed to actually does this, written by one of the pillars of the movement. (Negatively balanced means that pleasure does balance pain, but in a very unequal ratio. I think this is right, given our current brain states of representing pleasure much less vividly than pain).
Yes, I’m suggesting that neither you nor I can really imagine prolonged elevated pleasure states. Our current brain setup just doesn’t allow for them, again for evolutionary reasons.
So in sum, I still think pleasure and pain balance out when it comes to decision-making, and it’s just our current evolutionary wiring that makes suffering seem so much bigger than joy.
Does this assume there is some symmetry between the unimaginably bad outcomes and the unimaginably good outcomes?
It seems very clear to me that the worst outcomes are just so much more negative than the best outcomes are positive. I think that is just a fundamental aspect of how experience works.
Yes, and:
even if that’s true, the odds difference more than makes up for it The odds of a lot of people being tortured for eternity seems really small. The threat in a conflict with a compassionate AI is the only scenario I can think of where an AGI would do that. How likely is that? One in a million? A billion? And even in that case, is it going to really do it to a large number of people for a very long time? (That would imply that the threat failed, AND it won the conflict anyway, AND it’s going to follow through on the threat even though it no longer matters (but this isn’t probably important for overall odds so let’s not get hung up on this. The point is it’s a very specific scenario with low total odds).
The ratio between how good the best experiences are and how bad the worst pain is are maybe ten or a hundred times. Even people who’ve reported very bad pain that makes them want to die have been able to endure it for a long time. Similarly with the worst depressions.
So if we compare one in a million time one hundred (the worst estimates), we get one in ten thousand compated to maybe 50% of very very good long term outcomes. Expected pleasure is five thousand times (!) larger than expected suffering.
This is roughly a product of the fact that intelligent beings tend to want pleasure for themselves and each other. We’re trying to make aligned AGI. We’re not sure to succeed, but screwing it up so badly that we all get tortured is really unlikely. The few really bad sadists in the world aren’t going to get much say at all. So the odds are on are side, even though success is far from certain. Failure is much more likely to result in oblivion than torture. A good future is a “broad attractor” and a bad future is not.
it doesn’t need to stay that way.
That is a fundamental aspect of how experience works now. That’s also a result of evolution wiring us to pay more attention to bad things than good things.
That doesn’t need to stay how experience works. If we get the really good outcome, we get to re-wire our brains however we like. We could potentially be in a state of bliss while still thinking and doing stuff.
I appreciate the thoughtful response and that you seem to take the ideas seriously.
I do think it’s a fundamental aspect of how experience works, independently of how our brains are disposed to thinking about it, however I definitely think it’s possible to prophylactically shield or consciousness against the depths of suffering by modifying the substrate. I can’t tell whether we’re disagreeing or not.
I don’t know exactly how to phrase It, but I think a fundamental aspect of the universe is that as suffering increases in magnitude, it becomes less and less clear that there is (or can be) a commensurate value on the positive side which can negate it(trade off against it, even things out). I don’t think it’s true of the reverse.
Are you making the claim that this view is a faulty conclusion owing to the contingent disposition of my human brain?
Or are you making the claim that the disposition of my human brain can be modified so as to prevent exposure to the depths of suffering?
Thanks. I am indeed taking the ideas seriously.
This is getting more complex, and I’m running out of time. So I’ll be really brief here and ask for clarification:
I don’t understand why you think suffering is primary outside of particular brain/mind wiring. I hope I’m misunderstanding you. That seems wildly unlikely to me, and like a very negative view of the world.
So, clarify that?
Your intuition that no amount of pleasure might make up for suffering is the view of negative utilitarians. I’ve spent some time engaging with that worldview and the people who hold it. I think it’s deeply, fundamentally mistaken. It appears to be held by people who have suffered much more than they’ve enjoyed life. Their logic doesn’t hold up to me. If you think an entity disliking its experience (life) is worth avoiding, it seems like the simple inverse (enjoying life, pleasure) is logically worth seeking. The two cancel in decision-making terms.
So yes, I do think suffering seems primary to you based on your own intuitions and your own (very common) human anxiety, and the cold logic doesn’t follow that inution.
Yes, I’m definitely saying that your brain can be modified so that you experience more pleasure than suffering. To me it seems that thinking otherwise is to believe that your brain isn’t the whole of your experience. It is substance dualism, which has very little support in terms of either good arguments or good proponents. We are our brains, or rather the pattern within them. Change that pattern and we change our experience. This has been demonstrated a million times with brain injuries, drugs, and other brain changes. If dualism is true, the world is a massive conspiracy to make us think otherwise. If that’s the case, none of this matters, so we should assume and act as though materialism is true and we are our brains. If that’s the case, we can modify our experience as we like, given sufficient technology. AGI will very likely supply sufficient technology.
Thanks! No pressure to respond
Basically I think within the space of all possible varieties and extents of conscious experience, suffering starts to become less and less Commensurable with positive experience the further you go towards the extremes.
If option (A) is to experience the worst possible suffering for 100 years, prior to experiencing the greatest possible pleasure for N number of years, and option (B) is non existence, I would choose option (B), regardless of the value of N.
Should this count as evidence against their views? It seems clear to me that if you’re trying to understand the nature of qualitative states, first hand experience with extreme states is an asset.
I have personally experienced prolonged states of consciousness which were far worse than non-existence. Should that not play a part in informing my views? Currently I’m very happy, I fear death, I’ve experienced extraordinary prolonged pleasure states. Would you suggest I’m just not acquainted with levels of wellbeing which would cause me to meaningfully revaluate my view?
I think there’s also a sort of meta issue where people with influence are systematically less acquainted with direct experience of the extremes of suffering. Meaning that discourse and decision making will tend to systematically underweight experiences of suffering as a direct data source.
I agree with your last paragraph.
I’d also choose to not exist over the worst suffering for a hundred years—IF I was in my current brain-state. I’d be so insane as to be not-me after just a few minutes or hours if my synapses worked normally and my brain tried to adapt to that state. If I were forced to retain sanity and my character, it would be a harder choice if N got to be more than a hundred times longer.
Regardless, this intuition is just that. It doesn’t show that there’s something fundamentally more important about suffering than pleasure. Just that we’re better at imagining strong suffering than strong pleasure. Which is natural given the evolutionary incentives to focus us on pain.
I definitely didn’t mean to dismiss negative utilitarianism because some of the individuals who believe it seem damaged. I’m skeptical of it because it makes no sense to me, and discussions with NU people don’t help. The most rational among them slide back to negatively-balanced utilitarianism when they’re pressed on details—the FAQ I was pointed to actually does this, written by one of the pillars of the movement. (Negatively balanced means that pleasure does balance pain, but in a very unequal ratio. I think this is right, given our current brain states of representing pleasure much less vividly than pain).
Yes, I’m suggesting that neither you nor I can really imagine prolonged elevated pleasure states. Our current brain setup just doesn’t allow for them, again for evolutionary reasons.
So in sum, I still think pleasure and pain balance out when it comes to decision-making, and it’s just our current evolutionary wiring that makes suffering seem so much bigger than joy.