I think one of the points underrepresented in these “Not For the Sake of XXX Alone” posts is how people would respond to a least convenient world possible in which they would be forced to make sharp trade-offs between competing values.
For instance, I value diversity, a kind of narrative depth to raw experiences. But if I had to choose either sustainable, chemically induced unsophisticated pleasure or else diverse pain and misery with narrative depth, I’d almost certainly choose the pleasure.
This is relevant to FAI and CEV, I think. If the success probability of simple, pleasure-generating FAI is higher than more sophisticated (and difficult) “Not For the Sake of XXX Alone”-respecting FAI, it might be better opting for the pleasure-generating version.
I value diversity, a kind of narrative depth to raw experiences. But if I had to choose either sustainable, chemically induced unsophisticated pleasure or else diverse pain and misery with narrative depth, I’d almost certainly choose the pleasure.
Agreed. I also think people tend to underestimate the goodness of pure bliss: I have experienced such a state, and I’m here to tell you, the concerns about XXX become very much more minor than you would expect. They don’t disappear—if you like painting, you’ll still want to paint—but you suddenly understand how minor the pleasure painting gives you really is, in comparison.
The argument wasn’t that you need the joy of scientific discovery; it was that scientific discovery is important to us for reasons entirely apart from joy. You would never want a Soma substitute for scientific discovery, because that wouldn’t involve… you know… actual scientific discovery.
I’m reminded of Yudkowsky’s Not For the Sake of Happiness Alone.
I think one of the points underrepresented in these “Not For the Sake of XXX Alone” posts is how people would respond to a least convenient world possible in which they would be forced to make sharp trade-offs between competing values.
For instance, I value diversity, a kind of narrative depth to raw experiences. But if I had to choose either sustainable, chemically induced unsophisticated pleasure or else diverse pain and misery with narrative depth, I’d almost certainly choose the pleasure.
This is relevant to FAI and CEV, I think. If the success probability of simple, pleasure-generating FAI is higher than more sophisticated (and difficult) “Not For the Sake of XXX Alone”-respecting FAI, it might be better opting for the pleasure-generating version.
Agreed. I also think people tend to underestimate the goodness of pure bliss: I have experienced such a state, and I’m here to tell you, the concerns about XXX become very much more minor than you would expect. They don’t disappear—if you like painting, you’ll still want to paint—but you suddenly understand how minor the pleasure painting gives you really is, in comparison.
Or at least that’s how I felt, anyway.
He makes good points, but note that there’s nothing saying you couldn’t take Soma and participate in the joy of scientific discovery (or whatever).
The argument wasn’t that you need the joy of scientific discovery; it was that scientific discovery is important to us for reasons entirely apart from joy. You would never want a Soma substitute for scientific discovery, because that wouldn’t involve… you know… actual scientific discovery.
Additionally, another different take on this is Yvain’s Are Wireheads Happy?.