Unknown: “There is not at all the same intuitive problem here; it is much like the comparison made a while ago on Overcoming Bias between caning and prison time; if someone is given few enough strokes, he will prefer this to a certain amount of prison time, while if the number is continually increased, at some point he will prefer prison time.”
It may be a psychological fact that a person will always choose eventually. But this does not imply that those choices were made in a rationally consistent way, or that a rationally consistent extension of the decision procedures used would in fact involve addition of assigned utilities. Not only might decisions in otherwise unresolvable cases be made by the mental equivalent of flipping a coin, just to end the indecision, but what counts as unresolvable by immediate preference will itself depend on mood, circumstance, and other contingencies.
Similar considerations apply to speculations by moral rationalists regarding the form of an idealized personal utility function. Additivism and incommensurabilism both derive from simple moral intuitions—that harms are additive, that harms can be qualitatively different—and both have problems—determining ratios of badness exactly, locating that qualitative boundary exactly. Can we agree on that much?
Unknown: “There is not at all the same intuitive problem here; it is much like the comparison made a while ago on Overcoming Bias between caning and prison time; if someone is given few enough strokes, he will prefer this to a certain amount of prison time, while if the number is continually increased, at some point he will prefer prison time.”
It may be a psychological fact that a person will always choose eventually. But this does not imply that those choices were made in a rationally consistent way, or that a rationally consistent extension of the decision procedures used would in fact involve addition of assigned utilities. Not only might decisions in otherwise unresolvable cases be made by the mental equivalent of flipping a coin, just to end the indecision, but what counts as unresolvable by immediate preference will itself depend on mood, circumstance, and other contingencies.
Similar considerations apply to speculations by moral rationalists regarding the form of an idealized personal utility function. Additivism and incommensurabilism both derive from simple moral intuitions—that harms are additive, that harms can be qualitatively different—and both have problems—determining ratios of badness exactly, locating that qualitative boundary exactly. Can we agree on that much?