If your utility function u was replaced by 3u,there would be no observable difference in your behavior. So which of these functions is declared real and goes on to the interpersonal summing? “The same factor for everyone” isn’t an answer, because if u_you doesn’t equal u_me “the same factor” is simply meaningless.
@Tomhs2
A < B < C < D doesn’t imply that there’s some k such that kA>D
Yes it does.
I think you’re letting the notation confuse you. It would imply that, if A,B,C,D where e.g. real numbers, and that is the context the “<”-sign is mostly used in. But Orders can exist on sets other then sets of numbers. You can for example sort (order) the telephone book alphabetically, so that Cooper < Smith and still there is no k so that k*Cooper>Smith.
@most people here:
A lot of confusion is caused by the unspoken premise that a moral system should sort outcomes rather then actions, so that it doesn’t matter who would do the torturing or speck-placing. Now for Eliezer that assumption is de fide, because otherwise the concept of a friendly AI (sharing our ends and choosing the unimportant-declared means with its superior intelligence) is meaningless. But the assumption contradicts basically everyone’s intuition. So why should it convince anyone not following Eliezer’s religion?
[Edit: fixed some typos and formating years later]
Moral systems (at least, consistent ones with social consequences) deal in intentions, not actions per se. This is why, for instance, we find a difference between a bank teller giving away bank money to a robber at gun point, and a bank teller giving away money in order to get back at their employer. Same action, but the intent in question is different. A moral system interested only in actions would be indifferent to this distinction.
Asking for a preference between two different states of affairs where uncertainty, ignorance and impotence are removed allows for an easy isolation of the intention component.
@Sean
If your utility function u was replaced by 3u,there would be no observable difference in your behavior. So which of these functions is declared real and goes on to the interpersonal summing? “The same factor for everyone” isn’t an answer, because if u_you doesn’t equal u_me “the same factor” is simply meaningless.
@Tomhs2
I think you’re letting the notation confuse you. It would imply that, if A,B,C,D where e.g. real numbers, and that is the context the “<”-sign is mostly used in. But Orders can exist on sets other then sets of numbers. You can for example sort (order) the telephone book alphabetically, so that Cooper < Smith and still there is no k so that k*Cooper>Smith.
@most people here:
A lot of confusion is caused by the unspoken premise that a moral system should sort outcomes rather then actions, so that it doesn’t matter who would do the torturing or speck-placing. Now for Eliezer that assumption is de fide, because otherwise the concept of a friendly AI (sharing our ends and choosing the unimportant-declared means with its superior intelligence) is meaningless. But the assumption contradicts basically everyone’s intuition. So why should it convince anyone not following Eliezer’s religion?
[Edit: fixed some typos and formating years later]
Moral systems (at least, consistent ones with social consequences) deal in intentions, not actions per se. This is why, for instance, we find a difference between a bank teller giving away bank money to a robber at gun point, and a bank teller giving away money in order to get back at their employer. Same action, but the intent in question is different. A moral system interested only in actions would be indifferent to this distinction.
Asking for a preference between two different states of affairs where uncertainty, ignorance and impotence are removed allows for an easy isolation of the intention component.
Does this answer your question?