The concept of marriage depends on my internals in that a different human might disagree about whether a couple is married, based on the relative weight they place on religious, legal, traditional, and common law conceptions of marriage. For example, after a Catholic annulment and a legal divorce, a Catholic priest might say that two people were never married, whereas I would say that they were. Similarly, I might say that two men are married to each other, and someone else might say that this is impossible. How quickly those arguments have faded away! I don’t think someone would use the same example ten years ago.
It seems like “human values” aren’t particularly reflective then? Like I could describe the behavioral properties of a species of animal, including what they value or don’t value.
A lot of the particulars of humans’ values are heavily reflective. Two examples:
A large chunk of humans’ terminal values involves their emotional/experience states—happy, sad, in pain, delighted, etc.
Humans typically want ~terminally to have some control over their own futures.
Contrast that to e.g. a blue-minimizing robot, which just tries to minimize the amount of blue stuff in the universe. That utility function involves reflection only insofar as the robot is (or isn’t) blue.
I think you can unroll any of the positive examples by references to facts about the speaker. To be honest, I don’t understand what is supposed to be so reflective about “actual human values”, but perhaps it’s that the ontology is defined with reference to fairly detailed empirical facts about humans.
Positive: “easy to understand”, “appealing”, “native (according to me) representation”
Negative: “apple”, “gluon”, “marriage”
The concept of marriage depends on my internals in that a different human might disagree about whether a couple is married, based on the relative weight they place on religious, legal, traditional, and common law conceptions of marriage. For example, after a Catholic annulment and a legal divorce, a Catholic priest might say that two people were never married, whereas I would say that they were. Similarly, I might say that two men are married to each other, and someone else might say that this is impossible. How quickly those arguments have faded away! I don’t think someone would use the same example ten years ago.
It seems like “human values” aren’t particularly reflective then? Like I could describe the behavioral properties of a species of animal, including what they value or don’t value.
But that leaves something out?
A lot of the particulars of humans’ values are heavily reflective. Two examples:
A large chunk of humans’ terminal values involves their emotional/experience states—happy, sad, in pain, delighted, etc.
Humans typically want ~terminally to have some control over their own futures.
Contrast that to e.g. a blue-minimizing robot, which just tries to minimize the amount of blue stuff in the universe. That utility function involves reflection only insofar as the robot is (or isn’t) blue.
I think you can unroll any of the positive examples by references to facts about the speaker. To be honest, I don’t understand what is supposed to be so reflective about “actual human values”, but perhaps it’s that the ontology is defined with reference to fairly detailed empirical facts about humans.