I don’t understand what it means to say that an agent who “values fairness” does better than another agent. If two agents have different value systems, how can you say that one does better than another? Regarding EY and the Prisoner’s Dilemma, I agree that EY is making that claim but I think he is also making the claim “and this is evidence that rational agents should use FDT”.
To your first point:
If two agents had identical utility functions, except for one or two small tweaks, it feels reasonable to ask “Which of these agents got more utility/actualized it’s values more?” This might be hard to actually formalize. I’m mostly running on the intuition that sometimes humans that are pretty similar might look at another and say, “It seems like this other person is getting more of what they want than I am.”
Fair enough. Though in this case the valuing fairness is a big enough change that it makes a difference to how the agents act, so it’s not clear that it can be glossed over so easily.
I don’t understand what it means to say that an agent who “values fairness” does better than another agent. If two agents have different value systems, how can you say that one does better than another? Regarding EY and the Prisoner’s Dilemma, I agree that EY is making that claim but I think he is also making the claim “and this is evidence that rational agents should use FDT”.
To your first point: If two agents had identical utility functions, except for one or two small tweaks, it feels reasonable to ask “Which of these agents got more utility/actualized it’s values more?” This might be hard to actually formalize. I’m mostly running on the intuition that sometimes humans that are pretty similar might look at another and say, “It seems like this other person is getting more of what they want than I am.”
Fair enough. Though in this case the valuing fairness is a big enough change that it makes a difference to how the agents act, so it’s not clear that it can be glossed over so easily.