Human beings do not have values that are provably aligned with the values of other human beings.
Sure, but we “happily” compromise. AI should be able to understand and implement the compromise that is overall best for everyone.
Any AI that does value something infinitely will not have human values
AI can value the “best compromise” infinitely :). But agreed nothing else.
I’m not sure what it would mean exactly to value the best compromise infinitely, since part of that compromise would be the refusal to accept a sufficiently bad Mugging, which implies a utility bound.
But if an AI can compromise on some fuzzy or simplified set if values, what happened to the full complexity and fragility of human value?
Why does the compromise have to be a function of simplified values? I don’t think I implied that.
Sure, but we “happily” compromise. AI should be able to understand and implement the compromise that is overall best for everyone.
AI can value the “best compromise” infinitely :). But agreed nothing else.
I’m not sure what it would mean exactly to value the best compromise infinitely, since part of that compromise would be the refusal to accept a sufficiently bad Mugging, which implies a utility bound.
But if an AI can compromise on some fuzzy or simplified set if values, what happened to the full complexity and fragility of human value?
Why does the compromise have to be a function of simplified values? I don’t think I implied that.