We can fix this by incorporating a history to the utility function. …
Or tl;dr: you are changing the utility function to fit it more closely to your experienced desires. Well, your utility function can be anything at all, it’s “not up for grabs”, as we say, which means that it is totally up for grabs by the person whose function it is.
But there’s a meta-issue here. If you have a moral theory, whether utilitarianism of any sort or something else, and you find it yields conclusions that you find morally repugnant, from what standpoint can you resolve the conflict?
You can’t just say “the theory says this, therefore it’s right,” because where did the theory come from? Neither can you say “my inner moral sense trumps the theory,” or what was the theory for? Modus ponens versus modus tollens. When a theory derived from your intuitions has implications that contradict your intuitions, how do you resolve the conflict? Where can you stand, to do so?
As Socrates might ask a modern Euthyphro, “are the injunctions of your theory good because they follow from the theory, or do they follow from the theory because they are good?” In the first case, whence the theory? In the second, whence the good?
Personally I just take this as a bit of intellectual entertainment. It’s fun to try to formalize moral intuitions and then look for the part where the formalization breaks, but that’s it—I don’t expect it to actually change anything about my behavior or anything like that.
Or tl;dr: you are changing the utility function to fit it more closely to your experienced desires. Well, your utility function can be anything at all, it’s “not up for grabs”, as we say, which means that it is totally up for grabs by the person whose function it is.
But there’s a meta-issue here. If you have a moral theory, whether utilitarianism of any sort or something else, and you find it yields conclusions that you find morally repugnant, from what standpoint can you resolve the conflict?
You can’t just say “the theory says this, therefore it’s right,” because where did the theory come from? Neither can you say “my inner moral sense trumps the theory,” or what was the theory for? Modus ponens versus modus tollens. When a theory derived from your intuitions has implications that contradict your intuitions, how do you resolve the conflict? Where can you stand, to do so?
As Socrates might ask a modern Euthyphro, “are the injunctions of your theory good because they follow from the theory, or do they follow from the theory because they are good?” In the first case, whence the theory? In the second, whence the good?
Personally I just take this as a bit of intellectual entertainment. It’s fun to try to formalize moral intuitions and then look for the part where the formalization breaks, but that’s it—I don’t expect it to actually change anything about my behavior or anything like that.