So… suppose hypothetically that it were analogously true of humans that our likelihood of voluntarily maximizing “Fun” is dependent on being in an environment in which our access to Fun appears primarily determined by chance and our own efforts, and in which we believe Fun may soon run out.
It seems to follow from that supposition that if an outside force (e.g., a superhuman FAI) wants to maximize the amount of Fun we have, while still respecting our agency, it has to create such an environment.
How does adding that hypothetical constraint affect the conclusions of the Fun Theory Sequence? Or does it?
How does such an environment differ from the world we actually live in? Or does it?
The second part I’m less sure of. It may already have happened iff the world as it is is consistent with what we’d expect such an entity to create, which is precisely what I’m wondering about.
My instinct is to say “nonsense; it could certainly create a world where irrevocable suffering and death aren’t quite so common, without violating that constraint”… a point that the Fun Theory Sequence also makes at some length, IIRC. But on thinking about it I’m not at all sure that it could without spoiling the scenario altogether.
For example: even in the world as it is, we are remarkably willing as a species to behave as though we were being taken care of by an all-powerful supernatural force; it might be that any significant further reduction in our suffering that wasn’t visibly our own doing would bring us over a tipping point where that willingness became a literally irresistable temptation.
Dunno. It seems as though it ought to be possible to make more specific predictions based on that speculation, perhaps even falsifiable ones, but thus far I’ve failed to get traction.
unless suffering entities do not have an internal experience and are only simulated to satisfy our perverse utility functions. It seems possible that the maximum fun a human type entity can have is being born in a time of scarcity and experiencing the transition to non-scarcity, in which case we should expect lots of such entities to be living in a simulation similar to our current world.
For example: even in the world as it is, we are remarkably willing as a species to behave as though we were being taken care of by an all-powerful supernatural force; it might be that any significant further reduction in our suffering that wasn’t visibly our own doing would bring us over a tipping point where that willingness became a literally irresistable temptation.
Major objection: historically, humans have behaved like that regardless of the absolute amount of suffering they were witnessing. We only evaluate the goodness of our world on a relative scale, according to the worst and the best conditions we’ve seen.
I’m sympathetic to this line of reasoning, but not entirely convinced of it. People raised in more abusive environments do seem to come out different from people raised in less abusive environments, for at least some spreads, and it’s not clear to me that their result is improved by being kept unaware of people who are better off. Though it might be.
Did a medieval townsman behave as though he/she was protected from the bad things that could happen any less than a modern middle-class person (whether religious or not)? I don’t think so.
Difficult question. From what I’ve read I’d expect medieval townsmen to be more aware of particular dangers than modern middle-class people: there’s a huge volume of medieval charms and prayers against thieves that’ve come down to us, for example, and I’m fairly certain that reflects an actual preoccupation. But it’s not clear to me that this would be any stronger in absolute terms than, say, modern culture’s fear of pedophiles.
In any case the baseline seems to have been within the same order of magnitude, while the everyday threat of violence or theft would have been multiple orders of magnitude higher in medieval society. On the other hand, the medieval threat landscape may also have been less stable, which I’d expect to represent a source of stress in its own right.
In any case the baseline seems to have been within the same order of magnitude, while the everyday threat of violence or theft would have been multiple orders of magnitude higher in medieval society.
Yeah, that’s exactly what struck me as unbelievable about Dave’s assertion.
Just to make sure I understand… the “assertion” we’re talking about is the possibility that “any significant further reduction in our suffering that wasn’t visibly our own doing would bring us over a tipping point where that willingness [to behave as though we were being taken care of by an all-powerful supernatural force] became a literally irresistable temptation”… yes?
And what you find unbelievable about this is that when you compare modern willingness to behave that way to medieval willingness to behave that way, you find that they’re roughly the same, but that when you compare modern threat level to medieval threat level, you find the medieval threat level is significantly higher. Yes?
If I got both of those right, I am confused. Can you unpack the relationship between those two assertions more precisely?
(Another objection: it sounds more and more like the Christian take on the problem of evil, which is to say: fuck the deity that thinks it’s best for us, we should take care of ourselves instead. Even if we should grow lazier and more complacent, real suffering has a higher moral cost than whatever qualities our life might lose.
Didn’t most people here agree that the society of Brave New World, whatever its flaws and vices, is better than the current one?)
I’m pretty sure I’m not altogether following your thought process here, but yes, insofar as “the Christian take on the problem of evil” is to posit that human nature is such that current evil is a necessary condition of ultimately improving the human condition, then this hypothetical sounds somewhat like it. (Of course, the Christian version also has to account for why an all-powerful creator God caused human nature to have that property, which this hypothetical does not.)
As for relative moral costs… (shrug) as I’ve said repeatedly in this thread, I don’t know. More precisely… I agree that in the world where we choose between (lazy, complacent, and non-suffering) and (non-lazy, non-complacent, and suffering) it might well be preferable to eliminate suffering at the cost of laziness and complacency. But in the world where we choose between (lazy, complacent, and non-suffering for eternity) and (non-lazy, non-complacent, and suffering for a while, followed by much higher levels of anti-suffering for eternity), it’s not clear the same conclusion arises.
So… suppose hypothetically that it were analogously true of humans that our likelihood of voluntarily maximizing “Fun” is dependent on being in an environment in which our access to Fun appears primarily determined by chance and our own efforts, and in which we believe Fun may soon run out.
It seems to follow from that supposition that if an outside force (e.g., a superhuman FAI) wants to maximize the amount of Fun we have, while still respecting our agency, it has to create such an environment.
How does adding that hypothetical constraint affect the conclusions of the Fun Theory Sequence?
Or does it?
How does such an environment differ from the world we actually live in?
Or does it?
Not rhetorical questions.
The first requirement:
suggests that a FAI would not tell us that it exists. In other words, the singularity may already have happened.
Agreed with the first part of that.
The second part I’m less sure of. It may already have happened iff the world as it is is consistent with what we’d expect such an entity to create, which is precisely what I’m wondering about.
My instinct is to say “nonsense; it could certainly create a world where irrevocable suffering and death aren’t quite so common, without violating that constraint”… a point that the Fun Theory Sequence also makes at some length, IIRC. But on thinking about it I’m not at all sure that it could without spoiling the scenario altogether.
For example: even in the world as it is, we are remarkably willing as a species to behave as though we were being taken care of by an all-powerful supernatural force; it might be that any significant further reduction in our suffering that wasn’t visibly our own doing would bring us over a tipping point where that willingness became a literally irresistable temptation.
Dunno. It seems as though it ought to be possible to make more specific predictions based on that speculation, perhaps even falsifiable ones, but thus far I’ve failed to get traction.
unless suffering entities do not have an internal experience and are only simulated to satisfy our perverse utility functions. It seems possible that the maximum fun a human type entity can have is being born in a time of scarcity and experiencing the transition to non-scarcity, in which case we should expect lots of such entities to be living in a simulation similar to our current world.
Major objection: historically, humans have behaved like that regardless of the absolute amount of suffering they were witnessing. We only evaluate the goodness of our world on a relative scale, according to the worst and the best conditions we’ve seen.
I’m sympathetic to this line of reasoning, but not entirely convinced of it. People raised in more abusive environments do seem to come out different from people raised in less abusive environments, for at least some spreads, and it’s not clear to me that their result is improved by being kept unaware of people who are better off. Though it might be.
Did a medieval townsman behave as though he/she was protected from the bad things that could happen any less than a modern middle-class person (whether religious or not)? I don’t think so.
Difficult question. From what I’ve read I’d expect medieval townsmen to be more aware of particular dangers than modern middle-class people: there’s a huge volume of medieval charms and prayers against thieves that’ve come down to us, for example, and I’m fairly certain that reflects an actual preoccupation. But it’s not clear to me that this would be any stronger in absolute terms than, say, modern culture’s fear of pedophiles.
In any case the baseline seems to have been within the same order of magnitude, while the everyday threat of violence or theft would have been multiple orders of magnitude higher in medieval society. On the other hand, the medieval threat landscape may also have been less stable, which I’d expect to represent a source of stress in its own right.
Yeah, that’s exactly what struck me as unbelievable about Dave’s assertion.
Just to make sure I understand… the “assertion” we’re talking about is the possibility that “any significant further reduction in our suffering that wasn’t visibly our own doing would bring us over a tipping point where that willingness [to behave as though we were being taken care of by an all-powerful supernatural force] became a literally irresistable temptation”… yes?
And what you find unbelievable about this is that when you compare modern willingness to behave that way to medieval willingness to behave that way, you find that they’re roughly the same, but that when you compare modern threat level to medieval threat level, you find the medieval threat level is significantly higher. Yes?
If I got both of those right, I am confused. Can you unpack the relationship between those two assertions more precisely?
(Another objection: it sounds more and more like the Christian take on the problem of evil, which is to say: fuck the deity that thinks it’s best for us, we should take care of ourselves instead. Even if we should grow lazier and more complacent, real suffering has a higher moral cost than whatever qualities our life might lose.
Didn’t most people here agree that the society of Brave New World, whatever its flaws and vices, is better than the current one?)
No, they don’t. Put “Brave New World” into the LessWrong Google box and see for yourself.
But I think you’re just asking a rhetorical question as a way of saying that you do think that, without actually saying so.
Um, I’m surprised. I definitely saw something to the effect of that.
I’m pretty sure I’m not altogether following your thought process here, but yes, insofar as “the Christian take on the problem of evil” is to posit that human nature is such that current evil is a necessary condition of ultimately improving the human condition, then this hypothetical sounds somewhat like it. (Of course, the Christian version also has to account for why an all-powerful creator God caused human nature to have that property, which this hypothetical does not.)
As for relative moral costs… (shrug) as I’ve said repeatedly in this thread, I don’t know. More precisely… I agree that in the world where we choose between (lazy, complacent, and non-suffering) and (non-lazy, non-complacent, and suffering) it might well be preferable to eliminate suffering at the cost of laziness and complacency. But in the world where we choose between (lazy, complacent, and non-suffering for eternity) and (non-lazy, non-complacent, and suffering for a while, followed by much higher levels of anti-suffering for eternity), it’s not clear the same conclusion arises.