I think the caveats will turn out to matter a lot. One of the things that human moral beliefs do, in practice, is give other humans some reasons to trust you. If I know that you are committed, for non-instrumental reasons, to avoid manipulating* me into changing my values, that gives me reasons to trust you. Conversely, if your moral view is that it’s legitimate to lie to people to make them do what you want, people will trust you less.
Obviously, people have incentives to lie about their true values. I think equally obviously, people are paying attention and looking hard for that sort of hypocrisy.
*This sentence is true for a range of possible expansions of “manipulating”.
My statement was more observational than ideal, though. Sure, a rational agent can be averse to manipulating other people (and humans often are too), because agents can care about whatever they want. But that doesn’t bear very strongly on how the language is used compared to the fact that in real-world usage I see people say things like “improved his morals” by only three standards: consistency, how much society approves, and how much you approve.
I’d agree with that. Maybe with some caveats, but generally yes.
I think the caveats will turn out to matter a lot. One of the things that human moral beliefs do, in practice, is give other humans some reasons to trust you. If I know that you are committed, for non-instrumental reasons, to avoid manipulating* me into changing my values, that gives me reasons to trust you. Conversely, if your moral view is that it’s legitimate to lie to people to make them do what you want, people will trust you less.
Obviously, people have incentives to lie about their true values. I think equally obviously, people are paying attention and looking hard for that sort of hypocrisy.
*This sentence is true for a range of possible expansions of “manipulating”.
My statement was more observational than ideal, though. Sure, a rational agent can be averse to manipulating other people (and humans often are too), because agents can care about whatever they want. But that doesn’t bear very strongly on how the language is used compared to the fact that in real-world usage I see people say things like “improved his morals” by only three standards: consistency, how much society approves, and how much you approve.