Jordan Peterson has come up in the rationalsphere before; SSCreviewed his recent book 12 Rules for Life, which caused me to read it; Jacobian wrote about The Jordan Peterson Mask, and Robin Hanson reviewed his major scholarly work, Maps of Meaning. A key line from Hanson’s review:
In sum, Peterson comes across as pompous, self-absorbed, and not very self-aware. But on the one key criteria by which such a book should most be judged, I have to give it to him: the book offers insight.
So this article is my attempt to distill the core insight I found in Maps of Meaning. One reason I titled this “abridged” is because Peterson gives excellent summaries of his sections, which I will often just reprint fully. “Translated” is because he goes about his case much differently than I would; understandable, given the difference between our audiences. Peterson spends much of the book establishing plausibility that many different cultures have similar myths, and explaining what they represent using his terminology, whereas I am not moved by archaic human universality; even if all ancient cultures believed that the Sun revolved around an unmoving Earth, I want to believe in modern astronomy. To the extent that his subject matter is human psychology, even if all ancient cultures had the same view of what humans were like, I want to focus on what WEIRD people are like. But, thankfully, Peterson is primarily making an argument for a better understanding of progress, not obedience to the past. First I’ll try to explain the mythic perspective, and then Peterson’s characterization of the human condition, and then some commentary.
WHAT A MYTH IS
The world can be validly construed as forum for action, or as place of things.
The former manner of interpretation—more primordial, and less clearly understood—finds its expression in the arts of humanities, in ritual, drama, literature, and mythology. The world as a forum for action is a place of value, a place where all things have meaning. This meaning, which is shaped as a consequence of social interaction, is implication for action, or—at a higher level of analysis—implication for the configuration of the interpretive schema that produces or guides action.
The latter manner of interpretation—the world as place of things—finds its formal expression in the methods and theories of science. Science allows for increasingly precise determination of the consensually validatable properties of things, and for efficient utilization of precisely determined things as tools (once the direction such use is to take has been determined, through application of more fundamental narrative processes).
No complete world-picture can be generated without use of both modes of construal. The fact that one mode is generally set at odds with the other means only that the nature of their respective domains remains insufficiently discriminated. Adherents of the mythological worldview tend to regard the statements of their creeds as indistinguishable from empirical “fact,” even though such statements were generally formulated long before the notion of objective reality emerged. Those who, by contrast, accept the scientific perspective—who assume that it is, or might become, complete—forget that an impassable gulf currently divides what is from what should be.
Some of the language here seems obscure on first reading—what exactly is meant by “forum for action,” or “consensually validatable”?--but eventually seemed sensible to me. By ‘forum for action,’ he means the agents that take actions have their worldview molded around the constraints of determining the right actions to take. To use one of Peterson’s later examples, consider a rat in a cage it has explored well. Add a foreign object to the cage, like a cube of iron; the rat will initially flee, then as no danger presents itself, it will cautiously approach and inspect the object, attempting to figure out what it is good for. Can it be eaten? Used as bedding? If this exploration reveals the object is useless, it will ignored, as it distracts from the bits of the environment that are relevant to deciding what actions to take.
But, of course, useless is a two-place word. The cube of iron may be useless_rat without being useless_vaniver; there might be useful actions I could do with that cube, like fiddle with it or toss it or decorate with it or throw it away or store it. This example so far has focused on possibilities in a way that made the values implicit—while the cube might be too large for the rat to swallow it, it’s small enough that I could, but I didn’t include that in my list of actions because none of my goals are advanced by eating raw iron.
So the mythological worldview is one where the motivational role of beliefs takes center stage. But do we really have to call this “myth”? People frequently talk about “narrative” in ways that capture this ‘forum for action’ or ‘motivational’ business. It seems like we could discard the word “myth”: calling it narrative might just be fine, and makes clearer the relevance of the myths that describe “where we are” and “how we got here” as opposed to just “how to behave.” For the rest of this post, I’ll use ‘myth’ and ‘narrative’ interchangeably, but default to ‘myth’ because it’s Peterson’s language.
There seem to be two important takeaways from this section of the book:
First, functional humans require a motivational worldview, and the type signature of a motivational worldview is different than the type signature of the outputs of science. That doesn’t mean scientific knowledge isn’t useful for building that motivational worldview, it just means that there’s a gap between the outputs of the scientific process and the inputs of your ‘tastes’ that has to be filled by something. Reading through Is Humanism A Religion Substitute with this lens, I come away with “yep, Humanism counts as a motivational worldview and can be understood through the lens of myth.” Reading through Raised in Technophilia, it’s easy to see how science (seeing the world as a place of things) is insufficient to reach that particular viewpoint, and how the mythological mode of thinking captures the cultural transmission that’s happening. Science just describes changes that are happening; it is technophilia that identifies those changes as progress and the people making them as the Good Guys.
Second, historical cultures operated prior to the formation of the materialist / reductionist / naturalist / empiricist paradigm, and so their claims will be misunderstood if parsed in the empiricist language instead of in the mythical language. The core job of cultures is to teach their members how to properly integrate into the society, and so their stories are about that, rather than about what actually happened or how the world actually works. Stories and myths serve as catalogs of situations and examples of how to behave (or how not to behave) in those situations.
So suppose we agree that people need ‘purpose’ or ‘meaning’ or something similar, and that the type signature for this is ‘narrative’ or ‘myth’ or something similar. So what?
ORDER AND CHAOS
Peterson is mostly concerned with the myth of the hero, and spends most of the book discussing it. This is actually somewhat remarkable, given my presentation of the last chapter: one might assume that the book is like the catalog of human universals, and so collects myths that cover all of the important behaviors. That’s closer to what’s happening in Twelve Rules for Life; each of the rules is associated with stories that are important to the meaning and interpretation of that rule, and the combination of rules paints a picture of something like a whole human being. In Maps of Meaning, Peterson is instead trying to point to the core psychological need for myth, and the core myth that addresses this psychological need. Peterson spends much of the later bit of the book on the atrocities of the twentieth century; his desire to understand how they were possible (and, ideally, preventable) led to his discovery of the importance of myth, in part because this core myth ties into his diagnosis. The myth, in brief:
The world begins in a state of undifferentiated unity or unconsciousness; everything is the same because differences don’t exist yet. Then there is a separation, into the known and predictable (“order”) and the unknown and unpredictable (“chaos”). Order is secure and familiar; Chaos is dangerous and promising. The forces of order become insufficient to withstand the forces of chaos. An exploratory process (“the hero”) meets with chaos, defeats it, and transmutes it into order.
This is, in some sense, a presentation of the human condition—beginning in ignorance, learning some things, becoming self-aware enough to notice the difference between what is known and what is unknown, and deliberately learning about the unknown despite risks and discomfort and change. The myth typically includes a history of the gods that created the world, creating a mirror between the creation of an individual’s map and the creation of the external territory. Even beyond the individual process of learning and development, it describes the cultural process of learning and development; the society has some orderly way of dealing with what it knows, but the universe is larger than the society and so may change unpredictably (from the society’s viewpoint), causing decay and despair (another typical element of these myths) and the hero, by grappling with chaos, reforms the society and restores order and prosperity. Peterson often uses literary, figurative, and mythical language, in a way that I’m trying to avoid doing here, in a way that possibly makes his point easier to miss. For example, he connects ‘order’ with masculinity and ‘chaos’ with femininity, because this is so often done in historical myths, and one can see the connections. The unknown, containing many possibilities, is like the mother of many children. Tiamat (the feminine dragon of chaos) or her equivalent is depicted as birthing many different monsters. But while referring to the cluster of order-known-masculine and chaos-unknown-feminine lets you see the similarities across myths, it makes the reductionist project of separating out the distinct elements harder. If we load the core myth of the hero with all of these different features, then we can’t really point to a specific psychological need that this matches instead of ‘the human condition.’ It is not clear to me how much Peterson sees this holism as critical, as opposed to simply a feature of accumulated traditions. When faced with the wisdom of millenia, it is somewhat arrogant to say “ah, this lesson that I inferred from this myth is the final lesson to be inferred from that myth.”
But to some extent, this is the problem of psychology and self-understanding. Once a piece of the puzzle moves from chaos to order, there is yet more chaos to incorporate. Peterson makes a big deal out of how the Egyptian heroic deity myth includes a detail that the Babylonian heroic deity myth doesn’t (and how the creation and inclusion of this detail was the product of heroic cultural activity). In the Babylonian tradition, the king is associated with Marduk, youngest of the gods, who defeats the world-threatening Tiamat and becomes king of the gods. In the Egyptian tradition, the pharaoh is associated with Horus, child of Osiris, who defeats the improper king Set and rescues his father from the underworld, and becomes king of the gods. Osiris, in this telling, is the cultural practices of the past, Set is the way in which those practices have become maladaptive, and Horus is the process of discernment and discovery that creates the cultural practices of the present in continuity with those of the past while adapting to changing circumstances.
Peterson elaborates on the details and incorporates more myths (including Christianity and alchemy), but this is enough to capture the basic worldview and how it seems fundamentally compatible with the rationalist worldview. But instead of discussing more myths, I’ll jump straight to his diagnosis of the 20th century.
THE HERO AND THE ADVERSARY
One of these “hostile brothers” or “eternal sons of God” is the mythological hero. He faces the unknown with the presumption of its benevolence—with the (unprovable) attitude that confrontation with the unknown will bring renewal and redemption. He enters, voluntarily, into creative “union with the Great Mother,” builds of regenerates society, and brings peace to a warring world.
The other “son of God” is the eternal adversary. This “spirit of unbridled rationality,” horrified by his limited apprehension of the conditions of existence, shrinks from contact with everything he does not understand. This shrinking weakens his personality, no longer nourished by the “water of life,” and makes him rigid and authoritarian, as he clings desperately to the familiar, “rational,” and stable. Every deceitful retreat increases his fear; every new “protective law” increases his frustration, boredom, and contempt for life. His weakness, in combination with his neurotic suffering, engenders resentment and hatred for existence itself.
The personality of the adversary comes in two forms, so to speak—although these two forms are inseparably linked. The fascist sacrifices his soul, which would enable him to confront change on his own, to the group, which promises to protect him from everything unknown. The decadent, by contrast, refuses to join the social world, and clings rigidly to his own ideas—merely because he is too undisciplined to serve as an apprentice. The fascist wants to crush everything different, and then everything; the decadent immolates himself, and builds the fascist from his ashes. The bloody excesses of the twentieth century, manifest most evidently in the culture of the concentration camp, stand as testimony to the desires of the adversary and as monument to his power.
Here, “unbridled rationality” means something closer to the “high modernism” of Seeing Like a State. The core argument, as I understand it, is that totalitarianism grows from too much or too little confidence in knowledge. Both the overconfident and the underconfident have nothing left to learn; the first because they think they already know it (or it’s useless), and the second because they have abdicated their ability to learn. By nature, groups are associated with the known, rather than the process of the knower.
Peterson’s antidote is the heroic myth, as gradually constructed and refined by the slow accumulation of Western culture, which built up a myth of individual divinity / moral worth / moral judgment that involves a weird combination of humility and confidence—the sort of humility that allows one to actually adapt to reality, and the sort of confidence that allows one to stand up to the group or make mistakes while venturing into the unknown.
While a nice story (especially because it prescribes things I like), I find myself not convinced of its relevance or success. It seems like an important part of the story of Peterson’s life, and useful to include as context for why things are shaped particular ways (“ah, that’s why this aspect of the adversary is emphasized and that other bit minimized; this is the view of the adversary one gets by looking at the Nazis and Soviets.”). I also suspect that people should attempt to embody the heroic myth. But I suspect systemic problems have systemic solutions—the hero doesn’t just perfect themselves, they reorder society, and Peterson’s attempt to heroically reorder society in a way that causes more people to be heroes (because they have a crisper, more accessible view of what heroism means) seems possibly mismatched to the problem. It seems like the question of “what is the optimal number of scientists?”—yes, on the margin I would increase the amount of scientific thinking done by everyone, but how much I would pay to increase that for various people seems like the actual question.
It’s also not at all obvious that 20th century totalitarianism is incompatible with heroism as he sketches it out. Would a society in which more people set out to use their judgment to integrate their experience of their environment and the past be more tolerant and peaceful, or more revolutionary and violent?
CONCLUSION
Overall, the book was worth reading for me, and gave me a better sense of what ‘narrative’ is and what it’s useful for, but is not one I’d confidently recommend. The second quarter (“Mythological Representation: the Constituent Elements of Experience”) and last quarter (on alchemy) can probably be skipped, or heavily skimmed. Reading the summaries also seems like it captures much of the value of the book, and reading this summary perhaps captures much of the value of reading those summaries.
There seems to be a brand of scientific ennui which is somewhat common in the rationality community, where one focuses on predictions and models and ends up detached from the value in everyday life or pessimistic about one’s own prospects, which it seems like Peterson’s brand of mythological thinking is well-poised to counteract. As an example, the person who is convinced that the only cause that matters now is AI safety, but who is also convinced that they can’t be of any value to AI safety, seems at risk to see themselves as worthless and fall into depression. But this doesn’t help the project of AI safety, and it doesn’t help the individual in question. Contrast to ‘clean your room,’ which both makes the world more orderly (and the individual feel more powerful) on the object level, and teaches lessons about scale on the meta level (in that one should tackle challenges that are appropriately sized, and by doing so level up to be able to take on larger and more complicated challenges).
Continuing on the topic of AI alignment, it seems to me like thinking about narrative is useful in at least two ways. First, it seems to point towards the sort of self-awareness that’s necessary to encode ideas like corrigibility, and second, to the extent that alignment requires understanding humans, models of human psychology that more closely connect with what’s actually important to humans seem more likely to capture what’s important in life. For some reason, I feel more optimistic about an AI that has a good sense of what someone’s desired heroic journey is than an AI that has a good sense of what someone’s utility function is, in part because the first feels like it captures more meta-values and is less likely to be a snapshot of current opinions.
It seems to me like an important part of being a complete human is understanding the role narrative plays in one’s thinking, and how to take command of it. Myths, narratives, one’s taste and perception of the world as a place of fear and hope, danger and opportunity, and actions to be pursued or avoided, are all necessary parts of the human experience. The more advanced person does narrative to themselves, instead of having narrative done to them, embodying the role of the hero by incorporating their judgment into the process.
Maps of Meaning: Abridged and Translated
Jordan Peterson has come up in the rationalsphere before; SSC reviewed his recent book 12 Rules for Life, which caused me to read it; Jacobian wrote about The Jordan Peterson Mask, and Robin Hanson reviewed his major scholarly work, Maps of Meaning. A key line from Hanson’s review:
So this article is my attempt to distill the core insight I found in Maps of Meaning. One reason I titled this “abridged” is because Peterson gives excellent summaries of his sections, which I will often just reprint fully. “Translated” is because he goes about his case much differently than I would; understandable, given the difference between our audiences. Peterson spends much of the book establishing plausibility that many different cultures have similar myths, and explaining what they represent using his terminology, whereas I am not moved by archaic human universality; even if all ancient cultures believed that the Sun revolved around an unmoving Earth, I want to believe in modern astronomy. To the extent that his subject matter is human psychology, even if all ancient cultures had the same view of what humans were like, I want to focus on what WEIRD people are like. But, thankfully, Peterson is primarily making an argument for a better understanding of progress, not obedience to the past. First I’ll try to explain the mythic perspective, and then Peterson’s characterization of the human condition, and then some commentary.
WHAT A MYTH IS
Some of the language here seems obscure on first reading—what exactly is meant by “forum for action,” or “consensually validatable”?--but eventually seemed sensible to me. By ‘forum for action,’ he means the agents that take actions have their worldview molded around the constraints of determining the right actions to take. To use one of Peterson’s later examples, consider a rat in a cage it has explored well. Add a foreign object to the cage, like a cube of iron; the rat will initially flee, then as no danger presents itself, it will cautiously approach and inspect the object, attempting to figure out what it is good for. Can it be eaten? Used as bedding? If this exploration reveals the object is useless, it will ignored, as it distracts from the bits of the environment that are relevant to deciding what actions to take.
But, of course, useless is a two-place word. The cube of iron may be useless_rat without being useless_vaniver; there might be useful actions I could do with that cube, like fiddle with it or toss it or decorate with it or throw it away or store it. This example so far has focused on possibilities in a way that made the values implicit—while the cube might be too large for the rat to swallow it, it’s small enough that I could, but I didn’t include that in my list of actions because none of my goals are advanced by eating raw iron.
So the mythological worldview is one where the motivational role of beliefs takes center stage. But do we really have to call this “myth”? People frequently talk about “narrative” in ways that capture this ‘forum for action’ or ‘motivational’ business. It seems like we could discard the word “myth”: calling it narrative might just be fine, and makes clearer the relevance of the myths that describe “where we are” and “how we got here” as opposed to just “how to behave.” For the rest of this post, I’ll use ‘myth’ and ‘narrative’ interchangeably, but default to ‘myth’ because it’s Peterson’s language.
There seem to be two important takeaways from this section of the book:
First, functional humans require a motivational worldview, and the type signature of a motivational worldview is different than the type signature of the outputs of science. That doesn’t mean scientific knowledge isn’t useful for building that motivational worldview, it just means that there’s a gap between the outputs of the scientific process and the inputs of your ‘tastes’ that has to be filled by something. Reading through Is Humanism A Religion Substitute with this lens, I come away with “yep, Humanism counts as a motivational worldview and can be understood through the lens of myth.” Reading through Raised in Technophilia, it’s easy to see how science (seeing the world as a place of things) is insufficient to reach that particular viewpoint, and how the mythological mode of thinking captures the cultural transmission that’s happening. Science just describes changes that are happening; it is technophilia that identifies those changes as progress and the people making them as the Good Guys.
Second, historical cultures operated prior to the formation of the materialist / reductionist / naturalist / empiricist paradigm, and so their claims will be misunderstood if parsed in the empiricist language instead of in the mythical language. The core job of cultures is to teach their members how to properly integrate into the society, and so their stories are about that, rather than about what actually happened or how the world actually works. Stories and myths serve as catalogs of situations and examples of how to behave (or how not to behave) in those situations.
So suppose we agree that people need ‘purpose’ or ‘meaning’ or something similar, and that the type signature for this is ‘narrative’ or ‘myth’ or something similar. So what?
ORDER AND CHAOS
Peterson is mostly concerned with the myth of the hero, and spends most of the book discussing it. This is actually somewhat remarkable, given my presentation of the last chapter: one might assume that the book is like the catalog of human universals, and so collects myths that cover all of the important behaviors. That’s closer to what’s happening in Twelve Rules for Life; each of the rules is associated with stories that are important to the meaning and interpretation of that rule, and the combination of rules paints a picture of something like a whole human being. In Maps of Meaning, Peterson is instead trying to point to the core psychological need for myth, and the core myth that addresses this psychological need. Peterson spends much of the later bit of the book on the atrocities of the twentieth century; his desire to understand how they were possible (and, ideally, preventable) led to his discovery of the importance of myth, in part because this core myth ties into his diagnosis. The myth, in brief:
The world begins in a state of undifferentiated unity or unconsciousness; everything is the same because differences don’t exist yet. Then there is a separation, into the known and predictable (“order”) and the unknown and unpredictable (“chaos”). Order is secure and familiar; Chaos is dangerous and promising. The forces of order become insufficient to withstand the forces of chaos. An exploratory process (“the hero”) meets with chaos, defeats it, and transmutes it into order.
This is, in some sense, a presentation of the human condition—beginning in ignorance, learning some things, becoming self-aware enough to notice the difference between what is known and what is unknown, and deliberately learning about the unknown despite risks and discomfort and change. The myth typically includes a history of the gods that created the world, creating a mirror between the creation of an individual’s map and the creation of the external territory. Even beyond the individual process of learning and development, it describes the cultural process of learning and development; the society has some orderly way of dealing with what it knows, but the universe is larger than the society and so may change unpredictably (from the society’s viewpoint), causing decay and despair (another typical element of these myths) and the hero, by grappling with chaos, reforms the society and restores order and prosperity. Peterson often uses literary, figurative, and mythical language, in a way that I’m trying to avoid doing here, in a way that possibly makes his point easier to miss. For example, he connects ‘order’ with masculinity and ‘chaos’ with femininity, because this is so often done in historical myths, and one can see the connections. The unknown, containing many possibilities, is like the mother of many children. Tiamat (the feminine dragon of chaos) or her equivalent is depicted as birthing many different monsters. But while referring to the cluster of order-known-masculine and chaos-unknown-feminine lets you see the similarities across myths, it makes the reductionist project of separating out the distinct elements harder. If we load the core myth of the hero with all of these different features, then we can’t really point to a specific psychological need that this matches instead of ‘the human condition.’ It is not clear to me how much Peterson sees this holism as critical, as opposed to simply a feature of accumulated traditions. When faced with the wisdom of millenia, it is somewhat arrogant to say “ah, this lesson that I inferred from this myth is the final lesson to be inferred from that myth.”
But to some extent, this is the problem of psychology and self-understanding. Once a piece of the puzzle moves from chaos to order, there is yet more chaos to incorporate. Peterson makes a big deal out of how the Egyptian heroic deity myth includes a detail that the Babylonian heroic deity myth doesn’t (and how the creation and inclusion of this detail was the product of heroic cultural activity). In the Babylonian tradition, the king is associated with Marduk, youngest of the gods, who defeats the world-threatening Tiamat and becomes king of the gods. In the Egyptian tradition, the pharaoh is associated with Horus, child of Osiris, who defeats the improper king Set and rescues his father from the underworld, and becomes king of the gods. Osiris, in this telling, is the cultural practices of the past, Set is the way in which those practices have become maladaptive, and Horus is the process of discernment and discovery that creates the cultural practices of the present in continuity with those of the past while adapting to changing circumstances.
Peterson elaborates on the details and incorporates more myths (including Christianity and alchemy), but this is enough to capture the basic worldview and how it seems fundamentally compatible with the rationalist worldview. But instead of discussing more myths, I’ll jump straight to his diagnosis of the 20th century.
THE HERO AND THE ADVERSARY
Here, “unbridled rationality” means something closer to the “high modernism” of Seeing Like a State. The core argument, as I understand it, is that totalitarianism grows from too much or too little confidence in knowledge. Both the overconfident and the underconfident have nothing left to learn; the first because they think they already know it (or it’s useless), and the second because they have abdicated their ability to learn. By nature, groups are associated with the known, rather than the process of the knower.
Peterson’s antidote is the heroic myth, as gradually constructed and refined by the slow accumulation of Western culture, which built up a myth of individual divinity / moral worth / moral judgment that involves a weird combination of humility and confidence—the sort of humility that allows one to actually adapt to reality, and the sort of confidence that allows one to stand up to the group or make mistakes while venturing into the unknown.
While a nice story (especially because it prescribes things I like), I find myself not convinced of its relevance or success. It seems like an important part of the story of Peterson’s life, and useful to include as context for why things are shaped particular ways (“ah, that’s why this aspect of the adversary is emphasized and that other bit minimized; this is the view of the adversary one gets by looking at the Nazis and Soviets.”). I also suspect that people should attempt to embody the heroic myth. But I suspect systemic problems have systemic solutions—the hero doesn’t just perfect themselves, they reorder society, and Peterson’s attempt to heroically reorder society in a way that causes more people to be heroes (because they have a crisper, more accessible view of what heroism means) seems possibly mismatched to the problem. It seems like the question of “what is the optimal number of scientists?”—yes, on the margin I would increase the amount of scientific thinking done by everyone, but how much I would pay to increase that for various people seems like the actual question.
It’s also not at all obvious that 20th century totalitarianism is incompatible with heroism as he sketches it out. Would a society in which more people set out to use their judgment to integrate their experience of their environment and the past be more tolerant and peaceful, or more revolutionary and violent?
CONCLUSION
Overall, the book was worth reading for me, and gave me a better sense of what ‘narrative’ is and what it’s useful for, but is not one I’d confidently recommend. The second quarter (“Mythological Representation: the Constituent Elements of Experience”) and last quarter (on alchemy) can probably be skipped, or heavily skimmed. Reading the summaries also seems like it captures much of the value of the book, and reading this summary perhaps captures much of the value of reading those summaries.
There seems to be a brand of scientific ennui which is somewhat common in the rationality community, where one focuses on predictions and models and ends up detached from the value in everyday life or pessimistic about one’s own prospects, which it seems like Peterson’s brand of mythological thinking is well-poised to counteract. As an example, the person who is convinced that the only cause that matters now is AI safety, but who is also convinced that they can’t be of any value to AI safety, seems at risk to see themselves as worthless and fall into depression. But this doesn’t help the project of AI safety, and it doesn’t help the individual in question. Contrast to ‘clean your room,’ which both makes the world more orderly (and the individual feel more powerful) on the object level, and teaches lessons about scale on the meta level (in that one should tackle challenges that are appropriately sized, and by doing so level up to be able to take on larger and more complicated challenges).
Continuing on the topic of AI alignment, it seems to me like thinking about narrative is useful in at least two ways. First, it seems to point towards the sort of self-awareness that’s necessary to encode ideas like corrigibility, and second, to the extent that alignment requires understanding humans, models of human psychology that more closely connect with what’s actually important to humans seem more likely to capture what’s important in life. For some reason, I feel more optimistic about an AI that has a good sense of what someone’s desired heroic journey is than an AI that has a good sense of what someone’s utility function is, in part because the first feels like it captures more meta-values and is less likely to be a snapshot of current opinions.
It seems to me like an important part of being a complete human is understanding the role narrative plays in one’s thinking, and how to take command of it. Myths, narratives, one’s taste and perception of the world as a place of fear and hope, danger and opportunity, and actions to be pursued or avoided, are all necessary parts of the human experience. The more advanced person does narrative to themselves, instead of having narrative done to them, embodying the role of the hero by incorporating their judgment into the process.