IIRC, there was an earlier discussion which conceded the point that the human race will be happier in this scenario than in the scenario with no AI; the story depends on pumping the intuition that there’s some unrealized and undescribed third possibility which is so much better than either of those scenarios that choosing either of them constitutes a tragic ending.
IIRC, the author’s response to endorsing this scenario simply because in it people are happier without their opposite-sex partners (and because their opposite-sex partners are happier without them, as you say) was to mutter something deprecating about satisficers vs optimizers.
Full disclosure: I find this particular intuition pump leaves me cold, perhaps because I’m in a same-sex relationship. I’ve no doubt we could construct an analogous pump that would intuitively horrify me, and I might react differently.
The dubiously friendly AI, instead of creating artificial significant others, merely uses its computational ability to figure out that if it breaks up all existing relationships and instead puts people together with new partners, then everyone would be happier. Again, it then separates everyone in such a way that the existing partners could not get together in a reasonable amount of time. (You couldn’t do complete sex segregation, but you could put several pairs together on the same planet as long as the particular people who you broke them up with are on other planets.) Utopia or not?
And is it good or bad? There’s more than one objection to the original scenario and the point of this is 1) to separate them out, and 2) to make them more obvious.
Or a second scenario: The AI doesn’t try to create new relationships at all, whether with artificial or natural partners. Instead it just breaks up all relationships, and then wireheads everyone. It calculates that the utility gained from the wireheading is greater than the utility lost in breaking up the relationships. Is this good or bad?
(I would suggest that putting people with artificial partners is sort of like wireheading in that people might not want to be put under such circumstances, even though once they are already in such circumstances they might be happier.)
I would suggest that putting people with artificial partners is sort of like wireheading in that people might not want to be put under such circumstances, even though once they are already in such circumstances they might be happier.
This seems to stretch the notion of wireheading beyond usefulness. Many situations exist where we might endorse options retrospectively that we wouldn’t prospectively, whether through bias, limited information, random changes in perspective, or normal lack of maturity (“eew, girls have cooties!”). Relatively few of them rely on superstimuli or break our goal structure in a strong way.
IIRC, there was an earlier discussion which conceded the point that the human race will be happier in this scenario than in the scenario with no AI; the story depends on pumping the intuition that there’s some unrealized and undescribed third possibility which is so much better than either of those scenarios that choosing either of them constitutes a tragic ending.
IIRC, the author’s response to endorsing this scenario simply because in it people are happier without their opposite-sex partners (and because their opposite-sex partners are happier without them, as you say) was to mutter something deprecating about satisficers vs optimizers.
Full disclosure: I find this particular intuition pump leaves me cold, perhaps because I’m in a same-sex relationship. I’ve no doubt we could construct an analogous pump that would intuitively horrify me, and I might react differently.
Let’s modify the scenario a bit.
The dubiously friendly AI, instead of creating artificial significant others, merely uses its computational ability to figure out that if it breaks up all existing relationships and instead puts people together with new partners, then everyone would be happier. Again, it then separates everyone in such a way that the existing partners could not get together in a reasonable amount of time. (You couldn’t do complete sex segregation, but you could put several pairs together on the same planet as long as the particular people who you broke them up with are on other planets.) Utopia or not?
OK, I acknowledge receipt of this modified scenario.
And… what?
And is it good or bad? There’s more than one objection to the original scenario and the point of this is 1) to separate them out, and 2) to make them more obvious.
Or a second scenario: The AI doesn’t try to create new relationships at all, whether with artificial or natural partners. Instead it just breaks up all relationships, and then wireheads everyone. It calculates that the utility gained from the wireheading is greater than the utility lost in breaking up the relationships. Is this good or bad?
(I would suggest that putting people with artificial partners is sort of like wireheading in that people might not want to be put under such circumstances, even though once they are already in such circumstances they might be happier.)
This seems to stretch the notion of wireheading beyond usefulness. Many situations exist where we might endorse options retrospectively that we wouldn’t prospectively, whether through bias, limited information, random changes in perspective, or normal lack of maturity (“eew, girls have cooties!”). Relatively few of them rely on superstimuli or break our goal structure in a strong way.