Worth noting though that you should expect those who are most successful at getting more of what they want to have a utility function with a big overlap with the utility function of others, and to be able to credibly commit to not destroy value for other people. We live in a society.
Does what I want include wanting other people to get what they want? Often, yes. Not always. I want my family and friends and pets to get more of what they specifically want, and a large subset of strangers to mostly get more of what they want. But there are people who want to hurt others, and people who want to hurt themselves, and people who want things that reduce the world’s capacity to support people getting want they want, and I don’t want those people to get what they want. This will create conflict between agents, often value-destroying conflict, and zero or negative sum competitions, which I nevertheless will sometimes participate in and try to win.
Even if it destroys value for other people? If not, why not?
Yup. Rationality is orthogonal to values.
Worth noting though that you should expect those who are most successful at getting more of what they want to have a utility function with a big overlap with the utility function of others, and to be able to credibly commit to not destroy value for other people. We live in a society.
Or at least to be pretty good at appearing that way, while privately defecting to increase their own control of resources.
Does what I want include wanting other people to get what they want? Often, yes. Not always. I want my family and friends and pets to get more of what they specifically want, and a large subset of strangers to mostly get more of what they want. But there are people who want to hurt others, and people who want to hurt themselves, and people who want things that reduce the world’s capacity to support people getting want they want, and I don’t want those people to get what they want. This will create conflict between agents, often value-destroying conflict, and zero or negative sum competitions, which I nevertheless will sometimes participate in and try to win.