Wow, thank you so much. This is a lens I totally hadn’t considered.
You can see in the post how I was confused how evolution played a part in “imbuing” material terminal goals into humans. I was like, “but kinetic sculptures were not in the ancestral environment?”
It sounds like rather than imbuing humans with material goals, it has imbued a process by which humans create their own.
I would still define material goals as simply terminal goals which are not defined by some qualia, but it is fascinating that this is what material goals look like in humans.
This also, as you say, makes it harder to distinguish between emotional and material goals in humans, since our material goals are ultimately emotionally derived. In particular, it makes it difficult to distinguish between an instrumental goal to an emotional terminal goal, and a learned material goal created from reinforced prediction of its expected emotional reward.
E.g. the difference between someone wanting a cookie because it will make them feel good, and someone wanting money as a terminal goal because their brain frequently predicted that money would lead to feeling good.
I still make this distinction between material and emotional goals because this isn’t the only way that material goals play out among all agents. For example, my thermostat has simply been directly imbued with the goal of maintaining a temperature. I can also imagine this is how material goals play out in most insects.
Other emotions, like fear, anger, etc. are different. They can be thought of as “tilts”′ to our cognitive landscape. Even learning that we’re experiencing them is tricky. That’s why emotional awareness is a subject to learn about, not just something we’re born knowing. We need to learn to “feel the tilt”. Elevated heart rate might signal fear, anger, or excitement; noticing it or finding other cues are necessary to understand how we’re tilted, and how to correct for it if we want to act rationally. Those sorts of emotions “tilt the landscape” of our cognition by making different thoughts and actions more likely, like thoughts of how someone’s actions were unfair or physical attacks when we’re angry.
This makes a lot of sense. Yeah I was definitely simplifying all emotions to just their qualia effect, without considering their other physiological effects which define them. So I guess in this post when I say “emotion”, I really mean “qualia”.
But I’m pretty sure that predicted reward is pretty synonymous with what we call “values”.
Just to clarify, are you using “reward” here to also mean “positive (or a lack of negative) qualia”. Or is this reinforcement mechanism recursive by which we might learn to value something because of its predicted reward, but that reward is also a learned value.… and so on where the base case is an emotional reward. If so, how deep can it go?
Its worth mentioning that the majority of people are in camp #2. For example, in Illusionism as a Theory of Consciousness, Frankish notes:
Further, in the 2020 PhillPapers Survey, 998 English-speaking philosophers were asked “Hard problem of consciousness (is there one?): no or yes?” and 62% said yes while only 29% said no.
One other (more anecdotal) observation I have is that this distinction seems to often be at the crux of disagreement between moral realists and anti-realists. If you are in camp #2 and valenced qualia obviously exists, then it more naturally follows that qualia has inherent value and thus there is such thing as inherent value. However the r-value correlation between the two questions is only 0.21 so I may be missing something.
(also if you are a moral realist, I think there are important implications for AI safety worth being aware of.)