I conclude that you use “I don’t understand” here to indicate that you don’t find the reasoning compelling. I don’t find it compelling, either—hence, my not endorsing it—so I don’t have anything more to add on that front.
If those people propose that utility functions are timeless (e.g. the Mathematical Universe), or simply an intrinsic part of the quantum amplitudes that make up physical reality (is there a meaningful difference?), then under that assumption I agree. If beauty can be captured as a logical function then women.beautiful is right independent of any agent that might endorse that function. The problem of differing tastes, differing aesthetic value, that lead to sentences like “beauty is in the eye of the beholder” are a result of trying to derive functions by the labeling of relations. There can be different functions that designate the same label to different relations. x is R-related to y can be labeled “beautiful” but so can xSy. So while some people talk about the ambiguity of the label beauty and conclude that what is beautiful is agent-dependent, other people talk about the set of functions that are labeled as beauty-function or assign the label beautiful to certain relations and conclude that their output is agent-independent.
(nods) Yes, I think EY believes that rightness can be computed as a property of physical reality, without explicit reference to other agents.
That said, I think he also believes that the specifics of that computation cannot be determined without reference to humans. I’m not 100% clear on whether he considers that a mere practical limitation or something more fundamental.
I conclude that you use “I don’t understand” here to indicate that you don’t find the reasoning compelling. I don’t find it compelling, either—hence, my not endorsing it—so I don’t have anything more to add on that front.
If those people propose that utility functions are timeless (e.g. the Mathematical Universe), or simply an intrinsic part of the quantum amplitudes that make up physical reality (is there a meaningful difference?), then under that assumption I agree. If beauty can be captured as a logical function then women.beautiful is right independent of any agent that might endorse that function. The problem of differing tastes, differing aesthetic value, that lead to sentences like “beauty is in the eye of the beholder” are a result of trying to derive functions by the labeling of relations. There can be different functions that designate the same label to different relations. x is R-related to y can be labeled “beautiful” but so can xSy. So while some people talk about the ambiguity of the label beauty and conclude that what is beautiful is agent-dependent, other people talk about the set of functions that are labeled as beauty-function or assign the label beautiful to certain relations and conclude that their output is agent-independent.
(nods) Yes, I think EY believes that rightness can be computed as a property of physical reality, without explicit reference to other agents.
That said, I think he also believes that the specifics of that computation cannot be determined without reference to humans. I’m not 100% clear on whether he considers that a mere practical limitation or something more fundamental.