The Axiological Treadmill
The obvious reason that Moloch is the enemy is that it destroys everything we value in the name of competition and survival. But this is missing the bigger picture. We value what we value because, in our ancestral environment, those tended to be the things that helped us with competition and survival. If the things that help us compete and survive end up changing, then evolution will ensure that the things we value change as well.
To borrow a metaphor: Elua cheats. The hedonic treadmill has nothing on the axiological treadmill.
Consider a thought experiment. In Meditations on Moloch, Scott Alexander dreams up a dictatorless dystopia:
Imagine a country with two rules: first, every person must spend eight hours a day giving themselves strong electric shocks. Second, if anyone fails to follow a rule (including this one), or speaks out against it, or fails to enforce it, all citizens must unite to kill that person. Suppose these rules were well-enough established by tradition that everyone expected them to be enforced.
So you shock yourself for eight hours a day, because you know if you don’t everyone else will kill you, because if they don’t, everyone else will kill them, and so on. Every single citizen hates the system, but for lack of a good coordination mechanism it endures. From a god’s-eye-view, we can optimize the system to “everyone agrees to stop doing this at once”, but no one within the system is able to effect the transition without great risk to themselves.
Even if this system came into being ex nihilo it probably wouldn’t be stable in reality; a population that spends eight hours a day receiving strong shocks isn’t going to be able to feed itself, or reproduce. But assume for a moment that this system starts out economically and biologically stable (that is, people can still eat, and reproduce at the rate of replacement, despite the electric shocks, and that there are no outside countries ready to invade). What do we expect to happen over the long run?
Well, obviously there’s a strong evolutionary pressure to be tolerant to electric shocks. People who can tolerate those shocks better will do better on average than those who can’t. However, there’s another more subtle pressure at play: the pressure to ensure you shock yourself. After all, if you forget to shock yourself, or choose not to, then you are immediately killed. So the people in this country will slowly evolve reward and motivational systems such that, from the inside, it feels like they want to shock themselves, in the same way (though maybe not to the same degree) that they want to eat. Shocking themselves every day becomes an intrinsic value to them. Eventually, it’s no longer a dystopia at all.
They would be aghast at a society like ours, where Moloch has destroyed the value of receiving electrical shocks, all in the name of more perfect competition.
[Cross-posted from Grand, Unified, Empty.]
From my perspective
1. This has already happened
2. It’s still dystopian
No, it isn’t. What do I care which values evolution originally intended to align us with? What do I care which direction dysgenic pressure will push our values in the future? Those aren’t my values, and that’s all I need to know.
No, there is no selection pressure to shock yourself more than the required amount, anything beyond that is still detrimental to your reproductive fitness. Once we’ve evolved to barely tolerate the pain of electric shocks so as to not kill ourselves, the selection towards more pain tolerance stops, and people will still suffer a great deal because there is no incentive for evolution to fix pessimistic errors. You could perhaps engineer scenarios where humans will genuinely evolve to like a dystopia, but it certainly doesn’t happen by default, else suffering should already be a rare occurrence in nature.
Also, whenever someone suggests humans will evolve to do x it should make you shudder. The mountains of corpses and rivers of blood required for a species to evolve any feature makes “evolution” the most dastardly euphemism ever created.
I think that this kind of misrepresents the scale on which evolution happens—it’s not one generation, or two, it’s hundreds and thousands, and it’s taken relatively good care of the sources of suffering that are fundamental enough to persist and keep the selection pressure on across that time frame—we’re pretty good at not eating things that are toxic, breeding, avoiding predators and so on. The problem with evolution is that a significant number of sources of suffering are persistent enough to have a detrimental impact on an individual’s life, but transient enough to not be able to affect selection across generations.
Yes, that’s what pessimistic errors are about. I’m not sure what exactly you’re critiquing though?
Sorry if I wasn’t clear enough. My critique refers to your point about scenarios where humans evolve like a dystopia not being applicable because if it were, suffering should be a rare occurence—if I understand you correctly, you’re stating that if we could evolve to like dystopias, by this point in time we would have evolved to either avoid or like any source of suffering. My counterpoint to this is that there is a massive sub-multitude of sources of suffering that do not affect evolution in any way because they are too transient to effect any serious selection pressure.
I’m still confused about your critique, so let me ask you directly: In the scenario outlined by the OP, do you expect humans to eventually evolve to stop feeling pain from electrical shocks?
Eventually—sure. But for that eventuality to take place, the “electrical shock tyranny” would have to be more resilient than any political faction we’ve known of and persist for thousands of year. I doubt that this would be possible.
I guess I’m imagining that in a world where we only “barely tolerate” the pain, many people still try to defect over the long run due to psychological buildup, and end up getting killed. Thus there is some pressure to do better than “barely tolerate” at least. I’ve definitely updated to a model where the shock still produces suffering though; I no longer think the evolutionary pressure would be sufficient to remove the pain entirely.
The key is that evolution would try to find an equilibrium between the reproductive risk of not shocking yourself enough, and the reproductive risk of shocking yourself too much. Since not shocking yourself has much higher consequences on the margin, I expect evolution to bias slightly towards shocking yourself too much.
I disagree; if our physiology was already adapted to the shocks—our brain, heart and skeletal muscles were expecting these shocks—they’d be no longer harmful but probably necessary instead. Keeping a pain reaction or emotional aversion to these physiologically necessary shocks would be counterproductive; we should expect the link between electric shocks and pain to be broken eventually.
Suffering is not rare in nature because actually harmful things are common and suffering is an adequate response to them.
Evolution can’t dictate what’s harmful and what’s not; bigger peacock tails can be sexually selected for until it is too costly for survival, and an equilibrium sets in. In our scenario, since pain-inducing stimuli are generally bad for survival, there is no selection pressure to increase the pain threshold for electrical shocks after a certain equilibrium point. Because we start out with a nervous system that associates electrical shocks with pain, this pain becomes a pessimistic error after the equilibrium point and never gets fixed, i.e. humans still suffer under electrical shocks, just not so bad they’d rather kill themselves.
Why then is it possible to suffer pain worse than death? Why do people and animals suffer just as intensely beyond their reproductive age?
suffering after reproductive age is easy to answer—after reproductive age you are almost out of the reach of natural selection, especially e.g. some male wasps who keep living for a while after all the females have already mated and retreated into their nests. The males are now useless, nothing they do will change their reproductive success any more. So whatever happens to them doesn’t count—whether it’s a lot of suffering or the end of all suffering and infinite bliss. Unfortunately there’s no natural selection that would end their suffering after mating chances are depleted. Probably they’ll just keep having the same pain reactions to the same stimuli that they had before and during the mating season (because these already exist, not because they are necessary for anything). Probably they’ll have some additional old age problems accumulating just like old humans, because natural selection is not weeding these out.
Moloch is not always the enemy. Competition (among imperfectly-aligned agents) is the most efficient arbitration of different values. For almost everyone, survival is in fact something they value quite highly. Moloch happens when these pressures become so great (or are perceived as such) that they crowd out other values. Moloch destroys nothing except the illusion of freedom. Moloch creates value, just not the exact mix of value types that some or all participants would prefer.
“tended to be” and “are exactly and only” are very different statements. You’re saying the first, but your argument requires the second. My preferences as an individual human vary greatly from a historical human average. Even to the extent that they’re mutable and change with time, I have meta-preferences about the ways in which they change, and those are ALSO different from any historical or aggregate set.
Not even. There’s a bit of Slack even between genotype and phenotype, and a whole lot between biology and psychology.
I think of slack in this context like a “tax” on evolution—it doesn’t prevent the relevant forces (metaphorically supply and demand) from working, it just limits their speed, and prevents them from approaching a perfect solution with no inefficiency at the limits.