I tend to file this under “humans are stupid.” Messy creatures like ourselves undergo value drift, but decision-theoretically speaking, systems designed to optimize for some particular criterion have a natural incentive to keep that criterion. Cf. “The Basic AI Drives.”
It is probably best to model those as infections—or sometimes malfunctions.
Humans get infected with pathogens that make them do things like sneeze. Their values have not changed to value spreading snot on their neigbours, rather they are infected with germs—and the germs do value that.
It’s much the same with mind-viruses. A catholic conversion is best modelled as a memetic infection—rather than a genuine change in underlying values. Such people can be cured.
The fact that a change is reversible does not make it not real.
The fact that the final value system can be modeled as a starting value system modified by “memetic infection” does not make the final value system invalid. They are two different but equivalent ways of modelling the state.
Right. The point is that—under the “infection” analogy—people’s “ultimate” values change a lot less. How much they change depends on the strength of people’s memetic immune system—and there are some people with strong memetic immune systems whose values don’t change much at all.
Also, I suspect a lot of people who talk about how human values change are thinking of things, like aesthetics and preferred flavors of ice cream, that aren’t plausibly terminal values and that we often want to change over time.
I tend to file this under “humans are stupid.” Messy creatures like ourselves undergo value drift, but decision-theoretically speaking, systems designed to optimize for some particular criterion have a natural incentive to keep that criterion. Cf. “The Basic AI Drives.”
It is probably best to model those as infections—or sometimes malfunctions.
Humans get infected with pathogens that make them do things like sneeze. Their values have not changed to value spreading snot on their neigbours, rather they are infected with germs—and the germs do value that.
It’s much the same with mind-viruses. A catholic conversion is best modelled as a memetic infection—rather than a genuine change in underlying values. Such people can be cured.
The fact that a change is reversible does not make it not real.
The fact that the final value system can be modeled as a starting value system modified by “memetic infection” does not make the final value system invalid. They are two different but equivalent ways of modelling the state.
Right. The point is that—under the “infection” analogy—people’s “ultimate” values change a lot less. How much they change depends on the strength of people’s memetic immune system—and there are some people with strong memetic immune systems whose values don’t change much at all.
I’m not sure I follow you.
Are you saying that some agents change their values less often than others (or equivalently, are less likely to acquire “infections”)?
Also, I suspect a lot of people who talk about how human values change are thinking of things, like aesthetics and preferred flavors of ice cream, that aren’t plausibly terminal values and that we often want to change over time.