Inseparably Right; or, Joy in the Merely Good

Eliezer Yudkowsky9 Aug 2008 1:00 UTC
57 points
33 comments4 min readLW linkArchive

Followup to: The Meaning of Right

I fear that in my drive for full explanation, I may have obscured the punchline from my theory of metaethics. Here then is an attempted rephrase:

There is no pure ghostly essence of goodness apart from things like truth, happiness and sentient life.

What do you value? At a guess, you value the life of your friends and your family and your Significant Other and yourself, all in different ways. You would probably say that you value human life in general, and I would take your word for it, though Robin Hanson might ask how you’ve acted on this supposed preference. If you’re reading this blog you probably attach some value to truth for the sake of truth. If you’ve ever learned to play a musical instrument, or paint a picture, or if you’ve ever solved a math problem for the fun of it, then you probably attach real value to good art. You value your freedom, the control that you possess over your own life; and if you’ve ever really helped someone you probably enjoyed it. You might not think of playing a video game as a great sacrifice of dutiful morality, but I for one would not wish to see the joy of complex challenge perish from the universe. You may not think of telling jokes as a matter of interpersonal morality, but I would consider the human sense of humor as part of the gift we give to tomorrow.

And you value many more things than these.

Your brain assesses these things I have said, or others, or more, depending on the specific event, and finally affixes a little internal representational label that we recognize and call “good”.

There’s no way you can detach the little label from what it stands for, and still make ontological or moral sense.

Why might the little ‘good’ label seem detachable? A number of reasons.

Mainly, that’s just how your mind is structured—the labels it attaches internally seem like extra, floating, ontological properties.

And there’s no one value that determines whether a complicated event is good or not—and no five values, either. No matter what rule you try to describe, there’s always something left over, some counterexample. Since no single value defines goodness, this can make it seem like all of them together couldn’t define goodness. But when you add them up all together, there is nothing else left.

If there’s no detachable property of goodness, what does this mean?

It means that the question, “Okay, but what makes happiness or self-determination, good?” is either very quickly answered, or else malformed.

The concept of a “utility function” or “optimization criterion” is detachable when talking about optimization processes. Natural selection, for example, optimizes for inclusive genetic fitness. But there are possible minds that implement any utility function, so you don’t get any advice there about what you should do. You can’t ask about utility apart from any utility function.

When you ask “But which utility function should I use?” the word should is something inseparable from the dynamic that labels a choice “should”—inseparable from the reasons like “Because I can save more lives that way.”

Every time you say should, it includes an implicit criterion of choice; there is no should-ness that can be abstracted away from any criterion.

There is no separable right-ness that you could abstract from pulling a child off the train tracks, and attach to some other act.

Your values can change in response to arguments; you have metamorals as well as morals. So it probably does make sense to think of an idealized good, or idealized right, that you would assign if you could think of all possible arguments. Arguments may even convince you to change your criteria of what counts as a persuasive argument. Even so, when you consider the total trajectory arising out of that entire framework, that moral frame of reference, there is no separable property of justification-ness, apart from any particular criterion of justification; no final answer apart from a starting question.

I sometimes say that morality is “created already in motion”.

There is no perfect argument that persuades the ideal philosopher of perfect emptiness to attach a perfectly abstract label of ‘good’. The notion of the perfectly abstract label is incoherent, which is why people chase it round and round in circles. What would distinguish a perfectly empty label of ‘good’ from a perfectly empty label of ‘bad’? How would you tell which was which?

But since every supposed criterion of goodness that we describe, turns out to be wrong, or incomplete, or changes the next time we hear a moral argument, it’s easy to see why someone might think that ‘goodness’ was a thing apart from any criterion at all.

Humans have a cognitive architecture that easily misleads us into conceiving of goodness as something that can be detached from any criterion.

This conception turns out to be incoherent. Very sad. I too was hoping for a perfectly abstract argument; it appealed to my universalizing instinct. But...

But the question then becomes: is that little fillip of human psychology, more important than everything else? Is it more important than the happiness of your family, your friends, your mate, your extended tribe, and yourself? If your universalizing instinct is frustrated, is that worth abandoning life? If you represented rightness wrongly, do pictures stop being beautiful and maths stop being elegant? Is that one tiny mistake worth forsaking the gift we could give to tomorrow? Is it even really worth all that much in the way of existential angst?

Or will you just say “Oops” and go back to life, to truth, fun, art, freedom, challenge, humor, moral arguments, and all those other things that in their sum and in their reflective trajectory, are the entire and only meaning of the word ‘right’?

Here is the strange habit of thought I mean to convey: Don’t look to some surprising unusual twist of logic for your justification. Look to the living child, successfully dragged off the train tracks. There you will find your justification. What ever should be more important than that?

I could dress that up in computational metaethics and FAI theory—which indeed is whence the notion first came to me—but when I translated it all back into human-talk, that is what it turned out to say.

If we cannot take joy in things that are merely good, our lives shall be empty indeed.

Part of The Metaethics Sequence

Next post: “Sorting Pebbles Into Correct Heaps”

Previous post: “Morality as Fixed Computation”

Eliezer Yudkowsky9 Aug 2008 1:00 UTC
57 points
33 comments4 min readLW linkArchive