Conjuring An Evolution To Serve You

Eliezer Yudkowsky19 Nov 2007 5:55 UTC
75 points
26 comments4 min readLW linkArchive

GreyThumb.blog offers an interesting analogue between research on animal breeding and the fall of Enron. Before 1995, the way animal breeding worked was that you would take the top individual performers in each generation and breed from them, or their parents. A cockerel doesn’t lay eggs, so you have to observe daughter hens to determine which cockerels to breed. Sounds logical, right? If you take the hens who lay the most eggs in each generation, and breed from them, you should get hens who lay more and more eggs.

Behold the awesome power of making evolution work for you! The power that made butterflies—now constrained to your own purposes! And it worked, too. Per-cow milk output in the US doubled between 1905 and 1965, and has doubled again since then.

Yet conjuring Azathoth oft has unintended consequences, as some researchers realized in the 1990s. In the real world, sometimes you have more than animal per farm. You see the problem, right? If you don’t, you should probably think twice before trying to conjure an evolution to serve you—magic is not for the unparanoid.

Selecting the hen who lays the most eggs doesn’t necessarily get you the most efficient egg-laying metabolism. It may get you the most dominant hen, that pecked its way to the top of the pecking order at the expense of other hens. Individual selection doesn’t necessarily work to the benefit of the group, but a farm’s productivity is determined by group outputs.

Indeed, for some strange reason, the individual breeding programs which had been so successful at increasing egg production now required hens to have their beaks clipped, or be housed in individual cages, or they would peck each other to death.

While the conditions for group selection are only rarely right in Nature, one can readily impose genuine group selection in the laboratory. After only 6 generations of artificially imposed group selection—breeding from the hens in the best groups, rather than the best individual hens—average days of survival increased from 160 to 348, and egg mass per bird increased from 5.3 to 13.3 kg. At 58 weeks of age, the selected line had 20% mortality compared to the control group at 54%. A commercial line of hens, allowed to grow up with unclipped beaks, had 89% mortality at 58 weeks.

And the fall of Enron? Jeff Skilling fancied himself an evolution-conjurer, it seems. (Not that he, like, knew any evolutionary math or anything.) Every year, every Enron employee’s performance would be evaluated, and the bottom 10% would get fired, and the top performers would get huge raises and bonuses. Unfortunately, as GreyThumb points out:

“Everyone knows that there are many things you can do in any corporate environment to give the appearance and impression of being productive. Enron’s corporate environment was particularly conducive to this: its principal business was energy trading, and it had large densely populated trading floors peopled by high-powered traders that would sit and play the markets all day. There were, I’m sure, many things that a trader could do to up his performance numbers, either by cheating or by gaming the system. This gaming of the system probably included gaming his fellow traders, many of whom were close enough to rub elbows with.

“So Enron was applying selection at the individual level according to metrics like individual trading performance to a group system whose performance was, like the henhouses, an emergent property of group dynamics as well as a result of individual fitness. The result was more or less the same. Instead of increasing overall productivity, they got mean chickens and actual productivity declined. They were selecting for traits like aggressiveness, sociopathic tendencies, and dishonesty.”

And the moral of the story is: Be careful when you set forth to conjure the blind idiot god. People look at a pretty butterfly (note selectivity) and think: “Evolution designed them—how pretty—I should get evolution to do things for me, too!” But this is qualitative reasoning, as if evolution were either present or absent. Applying 10% selection for 10 generations is not going to get you the same amount of cumulative selection pressure as 3.85 billion years of natural selection.

I have previously emphasized that the evolution-of-foxes works at cross-purposes to the evolution-of-rabbits; there is no unitary Evolution God to praise for every beauty of Nature. Azathoth has ten million hands. When you conjure, you don’t get the evolution, the Maker of Butterflies. You get an evolution, with characteristics and strength that depend on your exact conjuration. If you just take everything you see in Nature and attribute it to “evolution”, you’ll start thinking that some cute little conjuration which runs for 20 generations will get you artifacts on the order of butterflies. Try 3.85 billion years.

Same caveat with the wonders of simulated evolution on computers, producing a radio antenna better than a human design, etcetera. These are sometimes human-competitive (more often not) when it comes to optimizing a continuous design over 57 performance criteria, or breeding a design with 57 elements. Anything beyond that, and modern evolutionary algorithms are defeated by the same exponential explosion that consumes the rest of AI. Yes, evolutionary algorithms have a legitimate place in AI. Consult a machine-learning expert, who knows when to use them and when not to. Even biologically inspired genetic algorithms with sexual mixing, rarely perform better than beam searches and other non-biologically-inspired techniques on the same problem.

And for this weakness, let us all be thankful. If the blind idiot god did not take a million years in which to do anything complicated, It would be bloody scary. 3.85 billion years of natural selection produced molecular nanotechnology (cells) and Artificial General Intelligence (brains), which even we humans aren’t going to get for a few more decades. If there were an alien demideity, morality-and-aesthetics-free, often blindly suicidal, capable of wielding nanotech and AGI in real time, I’d put aside all other concerns and figure out how to kill it. Assuming that I hadn’t already been enslaved beyond all desire of escape. Look at the trouble we’re having with bacteria, which go through generations fast enough that their evolutions are learning to evade our antibiotics after only a few decades’ respite.

You really don’t want to conjure Azathoth at full power. You really, really don’t. You’ll get more than pretty butterflies.

Eliezer Yudkowsky19 Nov 2007 5:55 UTC
75 points
26 comments4 min readLW linkArchive