[link] Whole Brain Emulation and the Evolution of Superorganisms

[Wei Dai]

Robin Hanson has made several recent posts on Overcoming Bias about upload economics. I remain mystified why he doesn’t link to or otherwise reference or comment on Carl Shulman’s 2010 paper, Whole Brain Emulation and the Evolution of Superorganisms, which mentions many of the same ideas and seems to have taken them to their logical conclusions. I was going to complain again in the comments section over there, but then I noticed that the paper hasn’t been posted or discussed here either. So here’s the abstract. (See above link for the full paper.)

Many scientists expect the eventual development of intelligent software programs capable of closely emulating human brains, to the point of substituting for human labor in almost every economic niche. As software, such emulations could be cheaply copied, with copies subsequently diverging and interacting with their copy-relatives. This paper examines a set of evolutionary pressures on interaction between related emulations, pressures favoring the emergence of superorganisms, groups of emulations ready to self-sacrifice in service of the superorganism. We argue that the increased capacities and internal coordination of such superorganisms could pose increased risks of overriding human values, but also could facilitate the solution of global coordination problems.

[Wei Dai]

I think this paper offers an object lesson in taking ideas seriously. I mean I thought I had taken the idea of uploads seriously, but apparently not and neither had anyone else. The conclusions reached in the paper do not seem to require really deep insight or information that is difficult to gather or process. Instead they seem like the obvious conclusions that anyone would reach if they just thought seriously about what would really happen once uploads became possible.

(Of course “obvious conclusions that anyone would reach if …” is an exaggeration, but I think you get my point. :)

[RobinHanson]

I posted comments on the paper here: http://​​www.overcomingbias.com/​​2011/​​10/​​shulman-on-superorgs.html

[Vladimir_Nesov]

(For the reference: these comments are discussed on LW here.)

[FiftyTwo]

That was very helpful. Thanks for thinking of people of the future.

[Kaj_Sotala]

This is an excellent (although scary) paper, much recommended.

[DanielVarga]

Robin has many-many clever and insightful observations, but I think the history books will only remember him for these unbelievably naive em posts. People of the future will find his idea of an em just as charmingly ridiculous as we see flapping-wing aeroplanes, and they will often quote him for comedic effect. At least that’s my hope, because I think he would deserve that for the hubris with which he builds these complex intellectual constructions on quicksand.

More seriously, this whole em series is another sterile example of what I called Individualism Bias at this old thread. Johnicolas’ short comment there points to the same direction as Carl’s paper, but from a different angle. (I am not an economist, but it seems to me that Robin failed to address Johnicolas’s points.)

[DanielVarga]

Robin has many-many clever and insightful observations, but I think the history books will only remember him for these unbelievably naive em posts. People of the future will find his idea of an em just as charmingly ridiculous as we see flapping-wing aeroplanes, and they will often quote him for comedic effect. At least that’s my hope, because I think he would deserve this for the hubris with which he builds these complex constructions on quicksand.

More seriously, this whole em series is a sterile example of what I called Individualism Bias at this old thread. I am not an economist, but it seems to me that Robin failed to address Johnicolas’s points there.

[CarlShulman]

Thanks for the plug!