Natural selection doesn’t produce “bad” outcomes, it produces expansionist, power-seeking outcomes, not at the level of the individual, but the whole system. In the meantime it will produce several intermediate states that may have adjacent even more power-seeking states, but are harder to find.
Humans developed several altruistic values because it was what produced the most fitness in the local search natural selection was running at the time. Cooperating with individuals from your tribe would lead to better outcomes than purely selfish behavior.
The modern world doesn’t make “evil” optimal. The reason violence has reduced is because negative-sum games among similarly capable individuals are an incredible waste of resources and we are undergoing selection in many different levels against that: violent people died in battle or were executed frequently during history, societies that enforced strong punishment against violence prospered more that ones that didn’t, cultures and religions that encouraged less harm made the groups that adopted them prosper more.
I’m not sure what about the OP or the linked paper would make you conclude anything you have concluded.
The reason we shouldn’t expect cooperation from AI is that it is remarkably more powerful than humans, and it may very well have better outcomes by paying the tiny cost of fighting humans if it can then turn all of us into more of it. I’m sure the pigs caged in our factory farms wouldn’t agree with your sense that the passage of time is favoring “goodness”.
There is also a huge asymmetry in AIs capability for self-modification, expansion and merging. In fact, I’d expect them to be less violent than humans among themselves, merging into single entities to avoid wasteful negative-sum competition, which is something that is impossible for humans to do.
One final thought: it may be that natural selection actually favors AI that cares more about humans than humans care about each other. Sound preposterous? Consider that there are species (such as Tasmanian devils) that present-day humans care about conserving but where the members of the species don’t show much friendliness to each other.
Regarding this, I don’t think it’s preposterous at all. It might be that initial cooperation with humans gives a head-start to the first AI which “locks-in” a cooperative value into it, and it carries it on even as it doesn’t need to. But longer term, I don’t know what would happen.
Natural selection doesn’t produce “bad” outcomes, it produces expansionist, power-seeking outcomes, not at the level of the individual, but the whole system. In the meantime it will produce several intermediate states that may have adjacent even more power-seeking states, but are harder to find.
Humans developed several altruistic values because it was what produced the most fitness in the local search natural selection was running at the time. Cooperating with individuals from your tribe would lead to better outcomes than purely selfish behavior.
The modern world doesn’t make “evil” optimal. The reason violence has reduced is because negative-sum games among similarly capable individuals are an incredible waste of resources and we are undergoing selection in many different levels against that: violent people died in battle or were executed frequently during history, societies that enforced strong punishment against violence prospered more that ones that didn’t, cultures and religions that encouraged less harm made the groups that adopted them prosper more.
I’m not sure what about the OP or the linked paper would make you conclude anything you have concluded.
The reason we shouldn’t expect cooperation from AI is that it is remarkably more powerful than humans, and it may very well have better outcomes by paying the tiny cost of fighting humans if it can then turn all of us into more of it. I’m sure the pigs caged in our factory farms wouldn’t agree with your sense that the passage of time is favoring “goodness”.
There is also a huge asymmetry in AIs capability for self-modification, expansion and merging. In fact, I’d expect them to be less violent than humans among themselves, merging into single entities to avoid wasteful negative-sum competition, which is something that is impossible for humans to do.
Regarding this, I don’t think it’s preposterous at all. It might be that initial cooperation with humans gives a head-start to the first AI which “locks-in” a cooperative value into it, and it carries it on even as it doesn’t need to. But longer term, I don’t know what would happen.