could you explain how we can resolve conflicts between two terminal values, if terminal means irreducible?
Suppose the following mind architecture:
When in a normal state, the mind desires games.
When the body reports low blood sugar levels, the mind desires food.
When in danger, the mind desires running away.
When in danger AND with low blood sugar levels, the mind desires freezing up.
Something like this has a system of resolving conflicts between terminal values: different terminal values are swapped in as the situation warrants. But although there is an evolutionary logic to them—their relative weights are drawn from the kind of a distribution which was useful for survival on average—the conflict-resolution system is not explicitly optimizing for any common currency, not even survival. There just happens to be a hodgepodge of situational variables and processes which end up resolving different conflicts in different ways.
I presented a more complex model of something like this in “Subagents, akrasia and coherence in humans”—there I did say that the subagents are optimizing for an implicit utility function, but the values for that utility function come from cultural and evolution-historical weights so it still doesn’t have any consistent “common currency”.
Often minds seem to end up at states where something like a particular set of goals or subagents ends up dominating, because those are the ones which have managed to accumulate the most power within the mind-system. This does not look like some of them became the most powerful through something like an appeal to shared values, but rather through just the details of how that person’s life-history, their personal neurobiological makeup, etc. happen to be set up and which kinds of neurological processes those details have happened to favor.
Similarly, governments repeat the same pattern at the intrapersonal level—value conflicts are not resolved through being weighted in terms of some higher-level value. Rather they are determined through a complex process where a lot of contingent details, such as a country’s parliamentary procedures, cultural traditions, voting systems etc. having a big influence on shaping which way the chips happen to fall WRT any given decision.
It occurred to me that, for a human being, there is no way not to make a choice between different preferences: in any next moment of time I do something, even continue to think or indulge in procrastination. I either eat, or run, so the conflict is always resolved.
However, an interesting thing is that sometimes a person tries to do two things simultaneously, for example, if content of the speech and the tone do not match. It has happened to me – and I had to explain that only content matter, and the tone should be ignored.
It occurred to me that, for a human being, there is no way not to make a choice between different preferences: in any next moment of time I do something, even continue to think or indulge in procrastination. I either eat, or run, so the conflict is always resolved.
This matches an excise you may be asked to do as part of Buddhist training towards enlightenment. During meditation, get your attention focused on itself, then try to do something other than what you would do. If you have enough introspective access, you’ll get an experience of being unable to do anything other than exactly what you do—you get first-hand experience with determinism at a level that bypasses the process that creates the illusion of free will. So not only can you only ever do the one thing you actually do (for some reasonable definition of what “one thing” is here), you can’t every do anything other than the one thing you end up doing, viz. there was no way any counterfactual was ever going to be realized.
Suppose the following mind architecture:
When in a normal state, the mind desires games.
When the body reports low blood sugar levels, the mind desires food.
When in danger, the mind desires running away.
When in danger AND with low blood sugar levels, the mind desires freezing up.
Something like this has a system of resolving conflicts between terminal values: different terminal values are swapped in as the situation warrants. But although there is an evolutionary logic to them—their relative weights are drawn from the kind of a distribution which was useful for survival on average—the conflict-resolution system is not explicitly optimizing for any common currency, not even survival. There just happens to be a hodgepodge of situational variables and processes which end up resolving different conflicts in different ways.
I presented a more complex model of something like this in “Subagents, akrasia and coherence in humans”—there I did say that the subagents are optimizing for an implicit utility function, but the values for that utility function come from cultural and evolution-historical weights so it still doesn’t have any consistent “common currency”.
Often minds seem to end up at states where something like a particular set of goals or subagents ends up dominating, because those are the ones which have managed to accumulate the most power within the mind-system. This does not look like some of them became the most powerful through something like an appeal to shared values, but rather through just the details of how that person’s life-history, their personal neurobiological makeup, etc. happen to be set up and which kinds of neurological processes those details have happened to favor.
Similarly, governments repeat the same pattern at the intrapersonal level—value conflicts are not resolved through being weighted in terms of some higher-level value. Rather they are determined through a complex process where a lot of contingent details, such as a country’s parliamentary procedures, cultural traditions, voting systems etc. having a big influence on shaping which way the chips happen to fall WRT any given decision.
It occurred to me that, for a human being, there is no way not to make a choice between different preferences: in any next moment of time I do something, even continue to think or indulge in procrastination. I either eat, or run, so the conflict is always resolved.
However, an interesting thing is that sometimes a person tries to do two things simultaneously, for example, if content of the speech and the tone do not match. It has happened to me – and I had to explain that only content matter, and the tone should be ignored.
This matches an excise you may be asked to do as part of Buddhist training towards enlightenment. During meditation, get your attention focused on itself, then try to do something other than what you would do. If you have enough introspective access, you’ll get an experience of being unable to do anything other than exactly what you do—you get first-hand experience with determinism at a level that bypasses the process that creates the illusion of free will. So not only can you only ever do the one thing you actually do (for some reasonable definition of what “one thing” is here), you can’t every do anything other than the one thing you end up doing, viz. there was no way any counterfactual was ever going to be realized.