there can be an irrational difference between what we believe our values are and what they really are.
Certainly. We are not transparent to ourselves: we have subconscious and situation-dependent drives; we don’t know in advance precisely how we’ll respond to hypothetical situations, how much we’ll enjoy and value them; we have various biases and inaccurate/fake memory issues which cause us to value things wrongly because we incorrectly remember enjoying them; our conscious selves self-deceive and are deceived by other brain modules; and so on.
Moreover, humans don’t have well-defined (or definable) utility functions; our different values conflict.
I’ve been trying to work through Torture versus Dustspecks and The Intuitions Behind Utilitarianism and getting stuck...
It seems Values are arational, but there can be an irrational difference between what we believe our values are and what they really are.
Certainly. We are not transparent to ourselves: we have subconscious and situation-dependent drives; we don’t know in advance precisely how we’ll respond to hypothetical situations, how much we’ll enjoy and value them; we have various biases and inaccurate/fake memory issues which cause us to value things wrongly because we incorrectly remember enjoying them; our conscious selves self-deceive and are deceived by other brain modules; and so on.
Moreover, humans don’t have well-defined (or definable) utility functions; our different values conflict.