The pattern, “Once you do X you won’t want to not do X” isn’t inherently evil. Once you breathe oxygen you won’t want to not breathe oxygen.
I think the deeper problem has to do with identity. If doing X implies that I will suddenly stop caring about everything I am doing, have done, or will do… is it still me?
Breathing oxygen isn’t a choice, though. You have to go to great lengths (such as inserting yourself into an environment where it isn’t breathable, such as vacuum or deep water) to stop breathing it for more than a few minutes before your conscious control is overriden.
a better example might be trying a hobby and finding you like it
But that sounds nice! Noone wants the wireheading to be nice! Its supposed to be scary but they want it anyway so its even scarier. People wanting fun stuff isn’t scary its just nice and its not interesting.
The correct response is the one at the intersection of possible responses to both cases such that the thread devolves into massive meta fun and uncertain pseudo-sarcasm.
The pattern, “Once you do X you won’t want to not do X” isn’t inherently evil. Once you breathe oxygen you won’t want to not breathe oxygen.
I think the deeper problem has to do with identity. If doing X implies that I will suddenly stop caring about everything I am doing, have done, or will do… is it still me?
The sunk cost fallacy may come into play as well.
“Once you stopped breathing oxygen you won’t want to breathe oxygen ever again.” is a more evil example.
Well, there is an adjustment period there.
Breathing oxygen isn’t a choice, though. You have to go to great lengths (such as inserting yourself into an environment where it isn’t breathable, such as vacuum or deep water) to stop breathing it for more than a few minutes before your conscious control is overriden.
You make a good argument that all those people who aren’t breathing are missing out :/
Seriously though, a better example might be trying a hobby and finding you like it so much you devote significant resources and time to it.
But that sounds nice! Noone wants the wireheading to be nice! Its supposed to be scary but they want it anyway so its even scarier. People wanting fun stuff isn’t scary its just nice and its not interesting.
Not sure if serious … ≖_≖
The correct response is the one at the intersection of possible responses to both cases such that the thread devolves into massive meta fun and uncertain pseudo-sarcasm.