...science bears upon instrumental issues but not terminal ones.
For what I consider non-obvious reasons, I disagree. As you say (and thanks for pointing this out explicitly),
What would falsify this idea would be an example of new factual knowledge changing someone’s perception of the moral value of some action, with this change persisting even after adjusting for the effect the knowledge has on the instrumental value of the action.
I have undergone changes in values that I would describe in this way. Namely, I had something I considered a terminal value that I stopped considering terminal upon realizing something factual about it. I’m guessing LucasSloan and Jayson_Virissimo are referring to similar experiences in thesecomments.
You could argue that it changing means “It wasn’t really terminal to begin with”. However, the separation of a given utility function into values and balancing operations is non-unique, so my current opinion is that the terminal/instrumental distinction is at best somewhat nominal. In other words, the change that it stopped feeling terminal may be the only sort of change worth calling “not being terminal anymore”.
So I think you should more precisely demand an example of a person’s utility function changing in response to knowledge. On the day of the factual realization I mentioned above, while it’s clear that my description of my utility function to myself and others changed, it’s not clear to me that the function itself changed much right away. But it does seem to me that over time, expressing it differently has gradually changed the function, though I can’t be sure.
I only hinted at all this when I added
First of all, you might change your morals in response to them not relating to each other in the way you expected. Ideas parse differently when they relate differently. “Teachers should be allowed to physically punish their students” might never feel the same to you after you find out it causes adult violence.
When I first made the utility function/description distinction, it was for abstract reasons (I was making a toy model of human morality for another purpose), and I didn’t quite notice the implications it would have for how people think of moral progress. Now in response to your demand for explicit examples, I’m a lot more motivated to sort this out. Thanks!
I have undergone changes in values that I would describe in this way. Namely, I had something I considered a terminal value that I stopped considering terminal upon realizing something factual about it.
Changing terminal values in response to learning is not only possible, but downright normal. We pursue one goal or another and find the life thus lived to be good or bad in our experience. We learn more about the goal-state or goal object, and it deepens or loses its attraction.
This needn’t mean that “the true terminal value” is pleasure or other positive emotion, even though happiness does play a role in such learning. Most people reject wire-heading: clearly pleasure is not their overarching “true terminal value.”
This needn’t mean that “the true terminal value” is pleasure or other positive emotion
True, it wouldn’t mean that pleasure was the actual terminal value, and the fact that many people reject wire-heading is evidence that pleasure is indeed not a terminal value for those people.
However, what role could “happiness” or feelings of well-being play, if not as true terminal values, if it’s in response to those feelings that people change (what they thought were) their terminal values?
For what I consider non-obvious reasons, I disagree. As you say (and thanks for pointing this out explicitly),
I have undergone changes in values that I would describe in this way. Namely, I had something I considered a terminal value that I stopped considering terminal upon realizing something factual about it. I’m guessing LucasSloan and Jayson_Virissimo are referring to similar experiences in these comments.
You could argue that it changing means “It wasn’t really terminal to begin with”. However, the separation of a given utility function into values and balancing operations is non-unique, so my current opinion is that the terminal/instrumental distinction is at best somewhat nominal. In other words, the change that it stopped feeling terminal may be the only sort of change worth calling “not being terminal anymore”.
So I think you should more precisely demand an example of a person’s utility function changing in response to knowledge. On the day of the factual realization I mentioned above, while it’s clear that my description of my utility function to myself and others changed, it’s not clear to me that the function itself changed much right away. But it does seem to me that over time, expressing it differently has gradually changed the function, though I can’t be sure.
I only hinted at all this when I added
When I first made the utility function/description distinction, it was for abstract reasons (I was making a toy model of human morality for another purpose), and I didn’t quite notice the implications it would have for how people think of moral progress. Now in response to your demand for explicit examples, I’m a lot more motivated to sort this out. Thanks!
Changing terminal values in response to learning is not only possible, but downright normal. We pursue one goal or another and find the life thus lived to be good or bad in our experience. We learn more about the goal-state or goal object, and it deepens or loses its attraction.
This needn’t mean that “the true terminal value” is pleasure or other positive emotion, even though happiness does play a role in such learning. Most people reject wire-heading: clearly pleasure is not their overarching “true terminal value.”
True, it wouldn’t mean that pleasure was the actual terminal value, and the fact that many people reject wire-heading is evidence that pleasure is indeed not a terminal value for those people.
However, what role could “happiness” or feelings of well-being play, if not as true terminal values, if it’s in response to those feelings that people change (what they thought were) their terminal values?