If you were a utilitarian, then why would you want to risk creating an AGI that had the potential to be an existential risk, when you could eliminate all suffering with the advent of WBE (whole brain emulation) and hence virtual reality (or digital alteration of your source code) and hence utopia?
That’s an enormous non sequitur. The resources necessary for maintaining a utopian virtual reality for a WBE may indeed be infinitesimal compared to those necessary for keeping a human happy. However, the easiness of multiplying WBEs is so great that it would rapidly lead to a Malthusian equilibrium, no matter how small the cost of subsistence per WBE might be.
For an in-depth treatment of this issue, see Robin Hanson’s writings on the economics of WBEs. (Just google for “uploads” and “ems” in the archives of Overcoming Bias and Hanson’s academic website.)
I’ll look into it. What is the motivation for these uploads to multiply? I can understand the human desire to. But even if uploads cannot directly change their source code, it seems pretty likely that they could change their utility function to something that is a little more logical (utilitarian). If they don’t have the desire to copy themselves indefinitely (something which humans basically have due to our evolutionary history), doesn’t this lower the probability of a population explosion uploads?
Clearly, there’s the immediate incentive to multiply uploads as cheap labor. Then there’s the fact that in the long run (possibly not even that long by our present standards), sheer natural selection will favor philoprogenitive inclinations, until it hits the Malthusian wall.
Cheap labor towards what end? Has the motivation of future uploads been addressed by Hanson? I think the true rejection is the fact that there’s an evolutionary advantage to mass replication. If there’s ever a scarcity of resources, the side with there could be a war or something and the side with fewer but smarter uploads would win and take the computing power from the mass replicators.
Cheap labor towards what end? Has the motivation of future uploads been addressed by Hanson?
He foresees a continuation of the property-based economy. (Which, if we discard naive and vague utopian thinking, is in fact an optimistic assumption given the realistic alternatives.)
Whether the uploads will themselves be property (i.e. slaves) or freely competing on the labor market, the obvious incentives will lead to them being multiplied until the marginal product of an upload is equal to the cost of the server space it occupies. (Which is to say, bare subsistence, thus leading to a Malthusian equilibrium.)
I think the true rejection is the fact that there’s an evolutionary advantage to mass replication. If there’s ever a scarcity of resources, the side with there could be a war or something and the side with fewer but smarter uploads would win and take the computing power from the mass replicators.
This sounds like an arbitrary story. What basis do you have for assuming such things would happen? And how would the community of “fewer but smarter” uploads avoid falling into its own tragedy of the commons where a subset of them defects by reproducing?
If it sounds arbitrary, it’s because I’m just speculating (which it seems most predictions about the FAR future are anyway). Not to say that all speculations are of equal merit.
My point isn’t necessarily about a war that could happen between uploads. I don’t assume that it will happen; just that it’s a possibility.
In evolution, it’s not the species that reproduces the most that survives. It’s the species with the most surviving members. Computational capacity concentrated in the hands of the few would enable greater coordination if there ever came to a fight over computation substrate. Although I don’t see what the motivation for this sort of war would be.
With uploads being nearly immortal, incredibly intelligent compared to modern humans, and able to digitally alter their hedonic set points, I don’t see how anything even moderately similar to modern society could persist after the invention of WBE. Why would an upload want to own a home or fight over property? They can just alter their hedonic-set point. Why would a WBE want to go to war? They aren’t going to be gaining any more happiness. They would just be needlessly endangering their lives. Why would they want to “work” hard? They don’t really need to pay for dentists, food, college, homes. What sort of work is being done in this economy? What are the motivations of the uploads to do this work? What are the motivations of the “employers” (if there still are employers)? The fact that you could alter you set-point and learn at an exponential rate, and live for what seems like an eternity makes me think that uploads will be nothing like humans today. But this is just my speculation, and I’m open to other ones.
You say (emphasis mine):
That’s an enormous non sequitur. The resources necessary for maintaining a utopian virtual reality for a WBE may indeed be infinitesimal compared to those necessary for keeping a human happy. However, the easiness of multiplying WBEs is so great that it would rapidly lead to a Malthusian equilibrium, no matter how small the cost of subsistence per WBE might be.
For an in-depth treatment of this issue, see Robin Hanson’s writings on the economics of WBEs. (Just google for “uploads” and “ems” in the archives of Overcoming Bias and Hanson’s academic website.)
I’ll look into it. What is the motivation for these uploads to multiply? I can understand the human desire to. But even if uploads cannot directly change their source code, it seems pretty likely that they could change their utility function to something that is a little more logical (utilitarian). If they don’t have the desire to copy themselves indefinitely (something which humans basically have due to our evolutionary history), doesn’t this lower the probability of a population explosion uploads?
Clearly, there’s the immediate incentive to multiply uploads as cheap labor. Then there’s the fact that in the long run (possibly not even that long by our present standards), sheer natural selection will favor philoprogenitive inclinations, until it hits the Malthusian wall.
Cheap labor towards what end? Has the motivation of future uploads been addressed by Hanson? I think the true rejection is the fact that there’s an evolutionary advantage to mass replication. If there’s ever a scarcity of resources, the side with there could be a war or something and the side with fewer but smarter uploads would win and take the computing power from the mass replicators.
He foresees a continuation of the property-based economy. (Which, if we discard naive and vague utopian thinking, is in fact an optimistic assumption given the realistic alternatives.)
Whether the uploads will themselves be property (i.e. slaves) or freely competing on the labor market, the obvious incentives will lead to them being multiplied until the marginal product of an upload is equal to the cost of the server space it occupies. (Which is to say, bare subsistence, thus leading to a Malthusian equilibrium.)
This sounds like an arbitrary story. What basis do you have for assuming such things would happen? And how would the community of “fewer but smarter” uploads avoid falling into its own tragedy of the commons where a subset of them defects by reproducing?
If it sounds arbitrary, it’s because I’m just speculating (which it seems most predictions about the FAR future are anyway). Not to say that all speculations are of equal merit.
My point isn’t necessarily about a war that could happen between uploads. I don’t assume that it will happen; just that it’s a possibility.
In evolution, it’s not the species that reproduces the most that survives. It’s the species with the most surviving members. Computational capacity concentrated in the hands of the few would enable greater coordination if there ever came to a fight over computation substrate. Although I don’t see what the motivation for this sort of war would be.
With uploads being nearly immortal, incredibly intelligent compared to modern humans, and able to digitally alter their hedonic set points, I don’t see how anything even moderately similar to modern society could persist after the invention of WBE. Why would an upload want to own a home or fight over property? They can just alter their hedonic-set point. Why would a WBE want to go to war? They aren’t going to be gaining any more happiness. They would just be needlessly endangering their lives. Why would they want to “work” hard? They don’t really need to pay for dentists, food, college, homes. What sort of work is being done in this economy? What are the motivations of the uploads to do this work? What are the motivations of the “employers” (if there still are employers)? The fact that you could alter you set-point and learn at an exponential rate, and live for what seems like an eternity makes me think that uploads will be nothing like humans today. But this is just my speculation, and I’m open to other ones.