On attunement

Joe Carlsmith25 Mar 2024 12:47 UTC
92 points
8 comments22 min readLW link

(Cross-posted from my website. Podcast version here, or search for “Joe Carlsmith Audio” on your podcast app.

This essay is part of a series that I’m calling “Otherness and control in the age of AGI.” I’m hoping that the individual essays can be read fairly well on their own, but see here for brief summaries of the essays that have been released thus far.)

“You, moon, You, Aleksander, fire of cedar logs.
Waters close over us, a name lasts but an instant.
Not important whether the generations hold us in memory.
Great was that chase with the hounds for the unattainable meaning of the world.”

~ Czeslaw Milosz, “Winter”

“Poplars (Autumn),” by Claude Monet (image source here)

My last essay examined a philosophical vibe that I (following others) call “green.” Green is one of the five colors on the Magic the Gathering Color Wheel, which I’ve found (despite not playing Magic myself) an interesting way of classifying the sort of the energies that tend to animate people.[1] The colors, and their corresponding shticks-according-to-Joe, are:

  • White: Morality.

  • Blue: Knowledge.

  • Black: Power.

  • Red: Passion.

  • Green: …

I haven’t found a single word that I think captures green. Associations include: environmentalism, tradition, spirituality, hippies, stereotypes of Native Americans, Yoda, humility, wholesomeness, health, and yin. My last essay tried to bring the vibe that underlies these associations into clearer view, and to point at some ways that attempts by other colors to reconstruct green can miss parts of it. In particular, I focused on the way green cares about respect, in a sense that goes beyond “not trampling on the rights/​interests of moral patients” (what I called “green-according-to-white”); and on the way green takes joy in (certain kinds of) yin, in a sense that contrasts with merely “accepting things you’re too weak to change” (what I called “green-according-to-black”).

In this essay, I want to turn to what is perhaps the most common and most compelling-to-me attempt by another color to reconstruct green—namely, “green-according-to-blue.” On this story, green is about making sure that you don’t act out of inadequate knowledge. Thus, for example: maybe you’re upset about wild animal suffering. But green cautions you: if you try to remake that ecosystem to improve the lives of wild animals, you are at serious risk of not knowing-what-you’re-doing. And see, also, the discourse about “Chesterton’s fence,” which attempts to justify deference towards tradition and the status quo via the sort of knowledge they might embody.

I think humility in the face of the limits of our knowledge is, indeed, a big part of what’s going on with green. But I think green cares about having certain kinds of knowledge too. But I think that the type of knowledge green cares about most isn’t quite the same as the sort of knowledge most paradigmatically associated with blue. Let me say more about what I mean.

How do you know what matters?

“I went out to see what I could see...”

~ Annie Dillard, “Pilgrim at Tinker Creek”

An 1828 watercolor of Tintern Abbey, by J.M.W. Turner (image source here)

Blue, to me, most directly connotes knowledge in the sense of: science, “rationality,” and making accurate predictions about the world. And there is a grand tradition of contrasting this sort of knowledge with various other types that seem less “heady” and “cognitive”—even without a clear sense of what exactly the contrast consists in. People talk, for example, about intuition; about system 1; about knowledge that lives in your gut and your body; about knowing “how” to do things (e.g. ride a bike); about more paradigmatically social/​emotional forms of intelligence, and so on.

And here, of course, the rationalists protest at the idea that rationality does not encompass such virtues (see, e.g., the discourse about “Straw Vulcans”). Indeed, if we understand “rationality” as a combination of “making accurate predictions” (e.g. “epistemic” rationality; cf blue) and “achieving your goals” (e.g., “instrumental” rationality; cf black), then an extremely broad variety of failures—e.g., social/​emotional clumsiness, indecision, over-thinking, disconnection from your intuition, falling-off-your-bike—can count as failures of rationality. With blue and black accounted for, then, is anything left over?

Well, yes—especially if we’re thinking of rationality as Yudkowsky does, in the context of the sort of meta-ethical anti-realism I discussed in “Deep atheism and AI risk.” In particular: I’ve written, previously, about the sense in which anti-realist rationality stumbles in the realm of ethics and value.

“Give anti-realist rationality a goal, and it will roar into life. Ask it what goals to pursue, and it gets confused. ‘Whatever goal would promote your goals to pursue?’ No, no, that’s not it at all.”

Or put another way: anti-realist rationality has a very rich concept of “instrumental rationality,” but a very impoverished concept of what we might call “terminal rationality”—that is, of how to do the “what matters intrinsically?” thing right. It tells you, at least, to not fail on the blue-and-black thing—to not form terminal goals based on a mistaken or incomplete picture of the world, or of what-will-lead-to-what. But beyond that, it goes silent.

Where, then, do your terminal goals come from? Well, for the most standard form of anti-realist rationality, from red. That is, from your heart, your desire, your passion—Hume’s famous slavemaster. That is, for all its associations with blue (and to a lesser extent, black), rationality (according to Yudkowsky) is actually, ultimately, a project of red. The explanatory structure is really: red (that is, your desires), therefore black (that is, realizing your desires), therefore blue (knowledge being useful for this purpose; knowledge as a form of power). Blue is twice secondary—a tool for black, which is itself a tool for red. (Of course, red can also value blue for its own sake—and perhaps this ultimately a better diagnosis of what’s going on with many rationalists. But from a philosophical perspective, intrinsically valuing knowledge is much more contingent.)

Indeed, in this sense, it’s not just green that anti-realist rationality struggles to capture. It’s also white—that is, morality. Anti-realist rationality has a concept of cooperation, in the sense of “getting-to-the-Pareto-frontier,” “making trade agreements,” and so on (with various fancy decision theories potentially playing a role in the process). But as I’ve written about previously, this sort of cooperation is too much a project of power to really capture morality—and in particular, it’s much too willing to kill, lie, defect, etc in interactions with weaker, dumber, and/​or unloved-by-the-powerful agents (this is core to why Yudkowsky doesn’t expect the AIs, for all their black-and-blue, to be nice to humans).[2]

And beyond this type of cooperation, what sort of white is left for anti-realist rationality? Just: whatever sort of white you happen to be red about. That is: morality is just one possible thing-your-heart-could-care-about, among many others. It’s another brand of paperclips. Should we have a color for paperclips as well? And for staples? And for staples-of-a-slightly-different-shape? And morality, too, comes in many different shapes. Which morality do we mean?

Indeed, for all the social connections between the Yudkowskian rationalists and the effective altruists, the philosophical connection, here, starts to break down. Effective altruism, as a philosophical project, tends to assume that there is this thing, “goodness,” which EAs try to maximize; or this thing, “altruism,” which EAs try to do effectively.[3] But Yudkowskian rationalism doesn’t, actually, have a privileged concept of “goodness,” or of “altruism” (see my essay “In search of benevolence” for more on this). Rather, there are a zillion concepts in the broad vicinity, which different hearts can latch onto differently—and it’s not clear what distinguishes them, deeply, from other sorts of goals or hobbies.

No wonder, then, that many of the philosophical founders of effective altruism (e.g. Singer, Parfit, Ord, MacAskill) tend towards moral realism. Effective Altruism is a lot about Morality with a capital M. Maybe it presents itself, in various contexts, as just-another-hobby. And sure, hobbyists are welcome. But various strands of philosophical EA want, underneath, to act with the righteousness of a True Church—to be doing, you know, the Good Thing, the Right Thing; and to be doing it the best way; the way you, like, should. Maybe you’re not obligated to do this (rather, it’s “supererogatory.”) And sure, you’re too weak to do it fully. But God smiles brighter as you do it more.

And this self-conception fits uncomfortably with treating white as ultimately grounded in red; morality as ultimately grounded in passion or sentiment. White wants God’s heart to smile on it; its own heart is beside the point, and lacks the authority white seeks.[4] That kind of authority, thinks paradigmatic white, needs to be more objective. It needs to speak with the world’s voice—a voice that says to the reflectively-coherent suffering-maximizers “you are wrong” and not just “you and I want different things, and I’m ready to fight about it.” And where does one go to call other people wrong? Standardly: to blue. That is, paradigmatic capital-M Morality wants its shtick to follow from (and be a form of) knowledge. Blue-therefore-white.[5] But anti-realism about meta-ethics denies morality this objectivity. Morality seeks grounding in blue; but red is the best it can get.

Right? Well, at some level: yes, probably. But I worry about telling the story too crudely, and in the wrong order. In particular: I worry that trying to ground ethics in either paradigmatically blue-style knowledge, or paradigmatically red-style passion, or in some combination, misses some other, more elusive dimension of normative epistemology—something neither paradigmatically red nor blue (even if, ultimately, it can be built out of red-and-blue); and something closely associated with wisdom. I’ll call this dimension “attunement.”

Gestures at attunement

“Don’t look upon the light in your eyes, look upon the sky.
And don’t feel the pain in your side, feel the wound there…
Don’t hear my words, hear the roughness and warmth of my mind.
Meet me here, face to face.”

~ Katja Grace, “As you know yourself”

What is attunement? I’m thinking of it, roughly, as a kind of meaning-laden receptivity to the world.[6] Something self-related goes quieter, and recedes into the background; something beyond-self comes to the fore. There is a kind of turning outwards, a kind of openness; and also, a kind of presence, a being in the world. And that world, or some part of it, comes forward as it always has been—except, often, strangely new, and shining with meaning.

Here’s a passage from Marilyn Robinson’s “Housekeeping” that evokes attunement for me:[7]

What was it like. One evening one summer she went out to the garden. The earth in the rows was light and soft as cinders, pale clay yellow, and the trees and plants were ripe, ordinary green and full of comfortable rustlings. And above the pale earth and bright trees the sky was the dark blue of ashes. As she knelt in the rows she heard the hollyhocks thump against the shed wall. She felt the hair lifted from her neck by a swift, watery wind, and she saw the trees fill with wind and heard their trunks creak like masts. She burrowed her hand under a potato plant and felt gingerly for the new potatoes in their dry net of roots, smooth as eggs. She put them in her apron and walked back to the house thinking, What have I seen, what have I seen. The earth and the sky and the garden, not as they always are. And she saw her daughters’ faces not as they always were, or as other people’s were, and she was quiet and aloof and watchful, not to startle the strangeness away.

Zadie Smith writes about another example. For much of her life, she hated the music of Joni Mitchell. It just sounded like noise: “a piercing sound, a sort of wailing.” Then, one day, she was visiting Tintern Abbey with her husband. He had Joni on in the background in the car. Smith hated it as always. They parked.

“I opened a car door onto the vast silence of a valley. I may not have had ears, but I had eyes. I wandered inside, which is outside, which is inside. I stood at the east window, feet on the green grass, eyes to the green hills, not contained by a non-building that has lost all its carved defenses… And then what? As I remember it, sun flooded the area; my husband quoted a line from one of the Lucy poems; I began humming a strange piece of music. Something had happened to me...”

Tintern Abbey (image source here)

Exactly what happened isn’t clear. But Smith’s experience of Joni Mitchell changes dramatically:

How is it possible to hate something so completely and then suddenly love it so unreasonably? How does such a change occur? … This is the effect that listening to Joni Mitchell has on me these days: uncontrollable tears. An emotional overcoming, disconcertingly distant from happiness, more like joy—if joy is the recognition of an almost intolerable beauty. It’s not a very civilized emotion.

Smith’s essay emphasizes the yin at stake in the attunement[8]—the listening, the letting-in—and also, the sense of recognizing something intensely (intolerably?) important, to which it is possible to be blind, or inadequately sensitive.[9] I’ve written about this before: “seeing more deeply,” “the doorway to real life.” I think experiences of beauty, spirituality, morality, and meaning all often involve a sense of attunement in this sense. And I think green cares a lot about that.

Indeed, what is Ogion trying to teach Ged, in silence, in the eyes of animals, and the flights of birds? The Wizard of Earthsea talks a lot about “true names”—but how do you learn them? Foster, in My Octopus Teacher, is trying to learn. And I think green-like figures of wisdom—Yoda, the Buddha, the archetype of an “elder”—often have very strong attunement vibes.

Admittedly, I’m painting in fairly broad strokes here. But hopefully, for present purposes, it’s enough of a gesture.

Attunement and your true heart

“You, music of my late years, I am called
By a sound and a color which are more and more perfect.

Do not die out, fire. Enter my dreams, love.
Be young forever, seasons of the earth.”

~ Czeslaw Milosz, “Winter”

“Hunters in the snow,” by Pieter Bruegel the Elder (image source here)

Now: when I wrote about attunement previously, under the heading of “seeing more deeply,” I said that it tends to pull me towards realism about value. This is centrally because it seems like it discloses something simultaneously beyond-myself and valuable/​important. That is, it has all the yin of blue—of knowledge, of receiving. But the thing-received, the thing-known, is something normative and meaningful.

Indeed, experiences of attunement are core to my own moral epistemology, and to my spirituality more generally. Philosophy, sure. But ultimately, for so many of us, it’s our deepest experiences that lead us onward. Some vision, some seeing, that says “this, this; don’t forget.” And said in some distinctive way; not as just-another-emotion, but with, it seems, some different depth—some particular harmony and clarity. For me, at least, this sort of depth is core to the weight and mystery and authority of that strange word, “goodness.” It’s related, I think, to the way sincerity feels like coming home; like something falling into its proper place. Chögyam Trungpa talks about “basic sanity.”

Does meta-ethical anti-realism preclude blue from receiving words like “goodness”? Blue alone: yes. And indeed, I expect that attunement will ultimately be a matter of both blue and red: of knowledge and love, your eyes and your heart, intermixed. But how do you see with your true heart’s eyes? Blue’s most paradigmatic answer is: “learn the facts; get ‘full information.’” But that doesn’t seem like it captures what’s going on with attunement very directly. In particular: experiences of attunement often feel much more like “realization” than like a change in belief. It’s often the same old facts; but with new resonance, new intensity, a new remembering.

And if we ask paradigmatic red to identify your true heart, it’s not clear that we capture attunement very well, either. In particular: paradigmatic red calls to mind a tumble of different passions and desires, colliding with each other in a contest of raw power—and the king of the hill gets to be Hume’s slavemaster.[10] Is your true heart, then, simply the strongest contestant, or coalition?[11] But at the least, insofar as the thing disclosed via attunement claims to be your true heart, it does not do so on the basis of felt intensity—or at least, not only. Hunger and lust, pride and fear—these can easily be more intense, at some level, than experiences of attunement. And they are quite a bit more common; quite a bit easier. Are they not, then, the truer red? Yet amidst all the shouting of ten thousand often-louder voices, when attunement speaks, the room goes quiet. And when attunement leaves, the room tries, so hard, to remember what it said, and to call it back again.

Of course, we can try to construct a story about your “true heart” that captures this dynamic. “I just do trust some experiences of care more than others. They just do leave a deeper and more sustained mark on my motivations and my orientation towards the world; and this is what makes them my true heart.” And ultimately, maybe something like this is the right story. That is, perhaps, for those of us for whom something like “attunement” plays a key role in shaping our core values (I don’t think this is everyone), this itself is centrally a fact about our particular pattern of care and meta-care; about how we do red.

If so, though, it seems like a very important fact to understand. Apparently, I trust certain types of experiences/​ways-of-being vastly more than others to shape what I do with my one and only life. Apparently, some experiences/​ways-of-being disclose something that is, to me, searingly and intolerably important. And this sort of experience seems to be associated, most centrally, not with paradigmatic red, or with paradigmatic blue, but with green—whatever that is.

Green, therefore...

“Quit your tents.”

~ Annie Dillard, “Teaching a Stone to Talk”

“Moses on Mount Sinai” by Jean-Léon Gérôme (image source here)

And even if green can/​should ultimately be built out of red and blue, we should make sure to tell the story in the right order. Here I think of a friend of mine, who identifies very strongly with morality, and with Effective Altruism. I told him my theory that paradigmatic Effective Altruism wants the story to be: knowledge, therefore morality. He said that for him, it feels like the story is more like: morality, therefore knowledge-therefore-morality. Or perhaps more accurately: morality, therefore: whatever it is such that therefore-morality. That is: the primary allegiance is to morality, whatever that is; whatever grounds it.[12] He is moralist, first; and meta-ethicist, second.[13]

I think something similar might be true for me (or parts of me), except with green first, instead—and in particular, green qua spirituality, green qua attunement. That is, I think the core story for me may be: green, therefore: whoa, that was important. How do I honor and do right by whatever that was? How do I see and respond to whatever I just saw-in-part? The earth and the sky and the garden. “The real world.” But how do you live there?

I do a lot of morality stuff. But a lot of it feels like green-therefore-white; morality as a way of honoring and responding to whatever-green-saw.

Of course, I do a lot of meta-ethics, too. I try to see green, too, more whole, and to figure out the right therefores—the true role of red, and of blue; the true nature of white. But the map is not the territory; I am more than my theory of myself; and my allegiance to seeing and responding to the world seen-by-attunement outstrips my confidence in any particular story about what grounds this allegiance, or of what-attunement-sees. This isn’t to say that blue can’t alter my attitude towards green (and green-without-blue isn’t real attunement, anyway). And no part of us needs to be the ultimate foundation—each can build and support and critique the others. But green, for me, is first and foremost according-to-itself: beauty, holiness, grace—raw and unrationalized.

Here I think of Robinson again:

Something happened, something so memorable that when I think back to the crossing of the bridge, one moment bulges like the belly of a lens and all the others are at the peripheries and diminished. Was it only that the wind rose suddenly, so that we had to cower and lean against it like blind women groping their way along a wall? or did we really hear some sound too loud to be heard, some word so true we did not understand it, but merely felt it pour through our nerves like darkness or water?

Too loud to be heard. How do you know it’s true if you can’t hear, or understand? But I think we should keep listening—keep taking attunement on its own terms—regardless. This is partly because I think we do understand the thing-attunement-sees, at some level—that’s why, I think, that goodness and beauty and holiness feel so much like coming home. “I had been my whole life a bell,” writes Annie Dillard, “and never knew it until at that moment I was lifted and struck.”[14] And it’s often thought that if something makes you-ring-like-a-bell, this is connected with stuff about your true heart. Hence the word “resonance.”

But also, to the extent we don’t yet have a settled story about what’s-up-with-attunement; to the extent blue does not yet grok green; still, I think that’s OK, too and that we should keep doing green-therefore-green in the meantime. In particular: when it comes to seeing with our true heart’s eyes, I think we should acknowledge how much we are still, yet, as blind women, groping our way. Anti-realists, at least, don’t actually have a clear story about their true hearts; and the realists don’t have the eyes they need to see God’s heart, either. But we still need to cross the bridge; and it is still extremely possible to fall. Indeed: the invention of AGI is a very big bridge. Maybe the biggest. And we might all fall at once.

It’s similar to what happens when we talk about “wisdom.” I say, often, that I want the future to be “wise.” But what does that mean? Again, doing blue-and-black right is a start, but it’s not enough; you have to get the ethics and meaning thing right, too. And how do we do that? We don’t know, we don’t know. We have scattered glimpses—histories; mistakes; lessons-learned. We have logic and empathy and imagination. We have traditions, archetypes, stories. And we have attunement. But we don’t have a settled program for becoming-wise. And we need to do it anyway.

To the extent green is the “wisdom” color, then, I think we should be pretty interested in making sure we’re staying in touch with green-on-its-own-terms. And indeed, when I think about the sort of wise that I want future people to be, I imagine them having the attunement thing in spades—some kind of intensity and tenderness and vastness of consciousness, some deep receptivity and responsiveness. If what one learns, from attunement, is “basic sanity,” I want the future to be sane.

“Waterloo Bridge,” by Claude Monet (image source here)

A future without attunement

“Who is there to carry on the life-thread of Wisdom?”

~ Hakuin

And how fragile is sanity? I have a friend, a moral realist,[15] who worries that the sort of Yudkowskian anti-realism pervasive amongst AI folks will create the world in its own image. That is, this sort of anti-realism assumes that the only type of agent you can build is a generalized paperclipper, a “Hume-bot,” chugging away in pursuit of its arbitrary preferences, instead of turning outwards towards the world, and seeking after some truer and deeper vision of meaning and morality. Aren’t we all Hume-bots, after all? That’s how the anti-realists model themselves, at least. So, worries my friend, the conditioners will make future agents—human and artificial—in their own self-image; blind, not just to the content of the Tao, but to the existence of the Tao; asking only, ever, about what they want, rather than about what’s right, what’s good, what’s worthy. And thus, the True Way will be lost forever; and the world will go blind.

I think my friend is too confident about moral realism (and/​or, too willing to wager on it). But I think he’s pointing at a real concern—and I think it’s a concern that anti-realists can share. To put it in my own terms: I think it’s a concern about a future that has lost attunement. Whatever our meta-ethics, we can agree that there is a thing that humans do, when they turn outwards, and with their hearts open; seeking, in yin, some truer contact with the good; trying to listen more deeply to the great song of the world. And we can agree that this thing, and the thing-it-discloses when done well, is profoundly precious. We want a future where it flowers fully; a future that sees in full, and with our whole hearts, what we now see only in part. Maybe the true story about this looks more like moral realism, or moral anti-realism; or, perhaps more likely, like neither in its current self-conception. But regardless, we want the future to cross the bridge, and with its soul intact. To finish, or to follow ever deeper, that most ancient pilgrimage: from cave to sun; from dream to the vast and waking world.

“Who is there to carry on the life-thread of Wisdom?” writes Hakuin. Who indeed. But if red-without-green grows unwise, then a future that runs only on red, or on red-therefore-black-therefore-blue, might lose the life-thread. Or to put it in more familiar Yudkowskian ontology: to the extent that whatever is going on with green, and with attunement, is itself core to our real red, our true hearts, then a future without attunement has made its heart false.

Indeed, I think we can read Lewis, in the Abolition of Man, as worried about something similar.[16] He wishes for a regenerate science such that, “while studying the It, it would not lose what Martin Buber calls the Thou-situation.” And attunement, to my mind, is closely related to approaching the world as a Thou—to that particular sort of yin that seeks, not just knowledge, but encounter; to give the world, the Other, its own dignity; to feel the weight of its being; be present with something else that is present, too. “Don’t look upon the light in your eyes, look upon the sky,” writes Katja Grace. “Meet me here, face to face.”

Of course, Lewis presents his concern, centrally, as about whether we will stay “within the Tao.” But if we think of the Tao less as The Objectively True Morality that All Cultures Have Basically Agreed On, and something more like “life lived from attunement,” then I start to feel better about passages like the following:

In the Tao itself, as long as we remain within it, we find the concrete reality in which to participate is to be truly human: the real common will and common reason of humanity, alive, and growing like a tree, and branching out, as the situation varies, into ever new beauties and dignities of application.

I do think that attunement participates in some concrete reality—something that draws us more deeply into our humanity, and into what I’ve called “real life.” And reframed in such terms, I think this passage actually gets at something pretty core; something that I very much want the age of AGI to stay “within,” and for anyone remotely nearby the power of a “conditioner” to remain especially in-contact-with. Indeed, it sounds a lot like bits of Yudkowsky’s own poetry about Coherent Extrapolated Volition. If we had grown up farther together. If we were more the people we wished we were. And when we imagine the path to good futures, I, at least, do actually imagine something akin to a civilization “alive and growing like a tree”—the way we’ve already been growing, painfully, over the centuries. A process that consists, centrally, not in the conditioners “figuring out the right values” and then “executing,” but rather in some kind of organic and ongoing self-adjustment; the way a plant grows, gradually, towards the light.[17]

“The Old Oak,” by Jules Dupre (image source here)

Primal blue

“Go where those others went to the dark boundary
for the golden fleece of nothingness your last prize

go upright among those who are on their knees
among those with their backs turned and those toppled in the dust

you were saved not in order to live
you have little time you must give testimony”

~ Zbigniew Herbert, “The Envoy of Mr. Cogito”

“Jason and Eros” by Gustave Moreau (image source here)

So far I’ve been talking about attunement centrally in terms of the normative stuff that it discloses. But various aspects of attunement also seem associated with non-normative types of knowledge—with a familiar sort of blue. For example: perception. To look upon the sky, rather than the light in your eyes, means to retain your grip not just on the raw data the perception provides, but on the function of perception—namely, to refer; to make contact; to see past the light to the thing-shining; to carve the right meaning from the noise. And whatever their other spiritual and normative connotations, ways of being in the vicinity of being present/​mindful/​”awake” seem to be doing something pretty blue as well—something directly related to the mundane (or at least, non-normative) truth.

That said: is it the same sort of mundane truth at stake when you make predictions, or improve your model of the world? I’m not sure. Certainly, failing to be “present” can easily lead to prediction-problems. But internally, various sorts of “presence” and “awakeness” often feel less propositional, and more like “getting a grip.” Like the same-old world coming into focus. “Poise.” It’s a type of blue related to that particular and especially-strange sort of knowledge that consciousness can have of itself—the thing that happens when, let’s just check one more time: yep, not-a-p-zombie.

Is that so basic? Descartes thought so. But the illusionists disagree. Regardless, though: the question burns hot and unanswered: what’s going on when your consciousness “knows itself”? Or relatedly, when you know that you exist, or that anything exists, or that you are some particular being among others? What sort of world looks out of any eyes—let alone, so many different and apparently-separate sets at once? Meet me here—but where is that?

Basic sanity is basic partly in the intimacy of its contact with this sort of primordial ground—Reality, being-a-thing, being-in-the-world, being-aware-at-all. It remains, at least to me, quite a mystery. In this sense, I associate attunement, not just with wisdom-qua-ethics, but with various other sorts of brute and not-understood-by-me sorts of existential awareness—what we might call green-blue; or maybe better, “primal blue.” Primal because I think there is something raw and animal-like about the way we know stuff like “I’m conscious” and “the world exists.” We know it before we know-what-we’re-knowing. We know it using the very foundations of our minds. No wonder, then, that it’s not going to win any prediction-markets—the foundations are priced in everywhere. But turn, directly, towards the foundations themselves, and they become a coal-face, and you start to touch raw rock.

Whether in ethics or elsewhere, I think that attunement is partly about this sort of living-at-the-coal-face; placing your mind, fully and openly, at its own edges; letting it propel itself towards the Real—that most-here, most-beyond—with its whole energy. And the coal face requires awareness that can exceed understanding; the ability to make contact with something not-mastered, not-understood. Some of this is about having the sort of map that lets the territory speak—that classic virtue of basic perception. But I think there’s also something else, related to being alive first, and making maps second, and in service. And of going, when necessary, without maps, to that dark boundary, where the others went. “Be your soul,” writes Katja Grace. “Press yourself against the world, into the world.”

Being your soul

“Beware of dryness of heart love the morning spring
the bird with an unknown name the winter oak

light on a wall the splendour of the sky
they don’t need your warm breath
they are there to say: no one will console you

be vigilant—when the light on the mountains gives the sign—arise and go
as long as blood turns in the breast your dark star

repeat old incantations of humanity fables and legends
because this is how you will attain the good you will not attain
repeat great words repeat them stubbornly
like those crossing the desert who perished in the sand”

Zbigniew Herbert, “The Envoy of Mr. Cogito”

“The Red Hills, Lake George,” by Georgia O’Keeffe (image source here)

And I think there is a connection between being our souls, in this sense, and keeping them intact as we cross the bridge into the age of AGI. Consider Lewis’s worry at the end of Abolition of Man.

The whole point of seeing through something is to see something through it. It is good that the window should be transparent, because the street or garden beyond it is opaque. How if you saw through the garden too? It’s no use trying to “see through” first principles. If you see through everything, then everything is transparent. But a wholly transparent world is an invisible world. To “see through” all things is the same as not to see.

What is this bit really about? Most of the book is about ethics—but the “first principles,” here, aren’t necessarily ethical. And indeed, I think Lewis is actually gesturing at (though: not explaining) a broader argument, made in e.g. Miracles, that Reason can’t view itself as a purely natural process, because (roughly) it is trying to follow universal laws of Reason, which is too deeply different from being subject to brute causation.[18] That is, Lewis thinks naturalism will not just kill the Tao; it will kill logic and math and all the other sorts of sanity (and hence, in his view, naturalism is insane). My impression is that various strands of both analytic and continental philosophy have similar worries about truth, reference, and other epistemic basics—that in becoming pure-causation, the world dissolves into a play of pure power. It’s not just that we lose contact with Objective Morality; we lose, also, the “contact” of truth and reference and encounter; and the only type of contact that remains is collision.

I’m not, generally, a fan of various arguments in this vicinity. Re: Lewis’s for example: I think that brute causal processes can themselves follow universal laws of Reason—see, e.g., a theorem-prover, or this marble adding machine—and that Reason can itself develop an empirical and naturalistic world-picture that validates that this is what’s happening with parts of our brain. And I suspect that we will be able to give similarly self-validating stories about stuff like reference as well, though I’ve thought less about the topic.

But I also think that Lewis’s argument, at least, is still pointing at something interesting re: “seeing through first principles”—and something related to being what Yudkowsky calls being “created already in motion.”[19] In particular: even though Reason is a brute causal process (though: not merely one), and can come to see (and validate) itself as such, the process of so-seeing requires being Reason—being your soul—as opposed to merely modeling it. You do, in fact, have to stay within something. You have to think—to seek, yourself, whatever it is that Reason seeks; to be the onrush of that part of your being. And this requires what I’ve previously called “living from the inside,” and “looking out of your own eyes,” instead of only from above. In that mode, your soul is, indeed, its own first principle; what Thomas Nagel calls the “Last Word.” Not the seen-through, but the seer (even if also: the seen).

Lewis seems to think that naturalists can’t do this (or: not consistently). That naturalists, by being too much within-the-world, have been somehow cast forever outside themselves. As I discussed in a previous essay, I think he’s wrong—and that his mistake, here, is closely related to why he seems to wrongly assume that naturalists must lose their grip on any non-crass ethics. Just as naturalists can be their reason and their logic, they can be the full richness of their values, too.

But just because naturalists can do this doesn’t mean they will. It is, in fact, strangely possible to not be our souls; to cut ourselves off from our full humanity; to live something other than real life. And if we demand that we be only enough soul as we have theory for, I fear we will leave too much of ourselves behind. This isn’t to belittle theory, or to sanctify mystery. But just as we can speak before we have a theory of reference; so too can we love past the edges of our theory-of-our-hearts, to the bird with the unknown name; the winter oak; the light on the wall; the splendor of the sky.

Indeed, how much of philosophy is this playing-catch-up, this struggling to understand something you already know, something you were doing-all-along? I love philosophy: but we can’t wait to catch up. We never could. But especially not while crossing this bridge, this desert, this new and daunting age. We need to use everything that any part of us knows about goodness and worthiness and holiness and justice. We need to be our souls fully; to carry the thread—even without knowing, fully, what we are carrying.

Herbert writes:

go because only in this way will you be admitted to the company of cold skulls
to the company of your ancestors: Gilgamesh Hector Roland
the defenders of the kingdom without limit and the city of ashes

Be faithful Go

Our city is not ashes yet. The blood still turns in our breasts. We are still chasing with the hounds—and I think the goal is attainable.

But the waters close over us. One of Robinson’s characters, a dying pastor, writes a letter to his young son, for the son to read when he grows up farther, and after his father is gone.

Theologians talk about a prevenient grace that precedes grace itself and allows us to accept it. I think there must also be a prevenient courage that allows us to be brave—that is, to acknowledge that there is more beauty than our eyes can bear, that precious things have been put into our hands and to do nothing to honor them is to do great harm. And therefore, this courage allows us, as the old men said, to make ourselves useful. It allows us to be generous, which is another way of saying exactly the same thing. But that is the pulpit speaking. What have I to leave you but the ruins of old courage, and the lore of old gallantry and hope? Well, as I have said, it is all an ember now, and the good Lord will surely someday breathe it into flame again.

… I’ll pray that you grow up a brave man in a brave country. I’ll pray you find a way to be useful.

(At least one more essay still left in the series.)

  1. ↩︎

    My relationship to the MtG Color Wheel is mostly via somewhat-reinterpreting Duncan Sabien’s presentation here, who credits Mark Rosewater for a lot of his understanding. What I say here won’t necessarily resonate with people who actually play Magic.

  2. ↩︎

    See Soares here for more.

  3. ↩︎

    Within constraints; with the part of their lives they choose to devote to this activity; etc…

  4. ↩︎

    This lack of authority is key to the intuitive pull of Lewis’s position, in the Abolition of Man, that anti-realists influencing the values of others must be tyrants.

  5. ↩︎

    And then, for EA, therefore-black-therefore-blue-again.

  6. ↩︎

    Webster’s dictionary for “attune” says: “(1) to bring into harmony, (2) to make aware or responsive.”

  7. ↩︎

    Indeed, I think part of what’s compelling about Robinson’s writing is her degree of attunement; the way the world, in her vision, seems to shine with quiet holiness; the way plain things appear somehow numinous. I associate Virginia Woolf with some quality in this vicinity, too—though of a different flavor than Robinson.

  8. ↩︎

    “I can’t listen to Joni Mitchell in a room with other people, or on an iPod, walking the streets. Too risky. I can never guarantee that I’m going to be able to get through the song without being made transparent—to anybody and everything, to the whole world. A mortifying sense of porousness. Although it’s comforting to learn that the feeling I have listening to these songs is the same feeling the artist had while creating them: ’At that period of my life, I had no personal defenses. I felt like a cellophane wrapper on a pack of cigarettes.”

  9. ↩︎

    Though I sympathize somewhat with Katja Grace, who finds that Smith’s essay as a whole doesn’t quite say what she wanted it to say.

    “I don’t think the words meant what I wanted them to mean, but it was arguably about what I wanted it to be about, and left me with the message I wanted. Which I somehow believe might be what she meant to mean, especially now that I try to find my own words. It sounded like she was saying, “if you lower your boundaries and give time to various initially unappealing art forms, they can be awesome”. But that’s a message in the wrong register. What I wanted it to say was, open yourself in some deep way, turn yourself around, open eyes that you didn’t know you had, and everything might touch you. Touch you like you are its edges and its texture and you know everything, even if you can’t put it into words—not just some heightened tendency to mindless tears, or another ‘positive mental state’ for the utility logs. Don’t ask for more reasons on your blind and empty abstracta table, be your soul instead, and press yourself against the world, into the world. Hear every cell itself, not the trace it leaves in your proposition set. ‘Attunement.’”

  10. ↩︎

    This is the psychological microcosm of Yudkowsky’s cosmic narrative.

  11. ↩︎

    Plus, presumably, a bunch of other idealization?

  12. ↩︎

    I have another friend, who also identifies very strongly with Morality, who thinks I shouldn’t be allowed to say that white’s shtick is “morality,” because all the other colors presumably think that their shtick is The Moral Thing, too. But I think she is wrong about what the other colors think.

  13. ↩︎

    Or maybe: not at all? Can we just ignore meta-ethics, please? Isn’t it a bit of a verbal dispute?

  14. ↩︎

    From Pilgrim at Tinker Creek.

  15. ↩︎

    Or at least, someone quite a bit more sympathetic to moral realism than me.

  16. ↩︎

    Like my friend above, Lewis focuses on realism vs. anti-realism about meta-ethics, but I don’t think we need to follow him in this.

  17. ↩︎

    From Christiano (2021):

    “I expect technology could radically transform the world on a timescale that would be disorienting to people, but for the most part that’s not how we want our lives to go in order to have the best chance of reaching the best conclusions about what to do in the long run. We do want some effects of technology --- we would like to stop being so hungry and sick, to have a little bit less reason to be at each other’s throats, and so on --- but we also want to be isolated from the incomprehensible, and to make some changes slowly and carefully.

    So I expect there to be a very recognizable thread running through humanity’s story, where many of the humans alive today just continue to being human and growing in a way that is familiar and comfortable, perhaps changing more quickly than we have in the past but never so quickly that we are at risk of losing our footing. The point of this is not because that’s how to have the best life (which may well involve incomprehensible mind-alteration or hyper-optimized virtual reality or whatever). It’s because we still have a job to do.

    The fact that you are able to modify a human to be much smarter does not mean that you need to, and indeed I think it’s important that you take that process slow. The kinds of moral change we are most familiar with and trust involve a bunch of people thinking and talking, gradually refining their norms and making small changes to their nature, raising new generations one after another.

    … I think that the community of humans taking things slowly and living recognizable lives isn’t an irrelevant sideshow that anyone serious would ignore in favor of thinking about the crazy stuff AI is doing “out there” (or the hyper-optimized experiences some of our descendants may immerse themselves in). I think there’s a real sense in which it’s the main thread of the human story; it’s the thread that determines our future and gradually expands to fill the universe.”

  18. ↩︎

    See this Wikipedia for more on this argument’s development in analytic philosophy, and see also Thomas Nagel’s The Last Word, which I think develops a somewhat similar line of thought. Here’s Lewis’s formulation in The Case for Christianity (though, I don’t think this is the strongest formulation of arguments in this vicinity, and I’ve tried to do a bit better in the main text):

    “Supposing there was no intelligence behind the universe, no creative mind. In that case, nobody designed my brain for the purpose of thinking. It is merely that when the atoms inside my skull happen, for physical or chemical reasons, to arrange themselves in a certain way, this gives me, as a by-product, the sensation I call thought. But, if so, how can I trust my own thinking to be true? It’s like upsetting a milk jug and hoping that the way it splashes itself will give you a map of London. But if I can’t trust my own thinking, of course I can’t trust the arguments leading to atheism, and therefore have no reason to be an atheist, or anything else. Unless I believe in God, I cannot believe in thought: so I can never use thought to disbelieve in God.”

  19. ↩︎

    See also “Where recursive justification hits bottom” and “My kind of reflection.”

Joe Carlsmith25 Mar 2024 12:47 UTC
92 points
8 comments22 min readLW link
Crossposted to EA Forum (0 points, 0 comments)