minds are behavior-executors and not utility-maximizers
I think it would be more accurate to say that minds are more accurately and simply modeled as behavior-executors than as utility-maximizers.
There are situations where the most accurate and simple model isn’t the one you want to use. For example, if I’m wanting to cooperate with somebody, one approach is to model them as a utility-maximizer, and then to search for actions that improve everybody’s utility. If I model them as a behavior-executors then I’ll be perceived as manipulative if I don’t get it exactly right.
Specifically, people don’t like to hear an explanation of my behavior of the form “Yes, I reasonably guessed that you would have preferred to get to the grocery store, and I knew that the only grocery store was to the south, but you were driving north so I helped you drive north.” Thus, if I model people as behavior-executors, I have a much more complicated game to play because I have to anticipate what they’ll do when they discover that I helped them to make a mistake.
That’s the difference between trying to save cute pandas and trying to rebalance a broken ecosystem.
To give somebody good advice, you need to understand what they actually want. Then you can introduce an idea into their mind for a behavior which will lead to them succeeding. The customer never knows what he really needs ; if he did, he wouldn’t be a customer.
I think it would be more accurate to say that minds are more accurately and simply modeled as behavior-executors than as utility-maximizers.
There are situations where the most accurate and simple model isn’t the one you want to use. For example, if I’m wanting to cooperate with somebody, one approach is to model them as a utility-maximizer, and then to search for actions that improve everybody’s utility. If I model them as a behavior-executors then I’ll be perceived as manipulative if I don’t get it exactly right.
Specifically, people don’t like to hear an explanation of my behavior of the form “Yes, I reasonably guessed that you would have preferred to get to the grocery store, and I knew that the only grocery store was to the south, but you were driving north so I helped you drive north.” Thus, if I model people as behavior-executors, I have a much more complicated game to play because I have to anticipate what they’ll do when they discover that I helped them to make a mistake.
That’s the difference between trying to save cute pandas and trying to rebalance a broken ecosystem.
To give somebody good advice, you need to understand what they actually want. Then you can introduce an idea into their mind for a behavior which will lead to them succeeding. The customer never knows what he really needs ; if he did, he wouldn’t be a customer.