But natural selection is what made humans nice. I wouldn’t argue that niceness is an inevitable outcome of any darwinian process—that would be a strawman. But the set of evolutionary pressures which gave rise to humans selected for individuals who were able to coexist in tribes, and this selection pressure produced, among other things, niceness as a general quality. At least for people we consider in our in-group.
It doesn’t even require an understanding of game theory. Non-psychopaths aren’t nice to others because they worked out the risk-reward calculations and determined niceness has the highest payoff. They’re nice because it’s nice to be nice, because that’s a feeling evolution selected for.
There’s no a priori reason why an AI can’t be evolved (*ahem*, “trained via reinforcement learning on human feedback”) to produce a similar drive for niceness.
But natural selection is what made humans nice. I wouldn’t argue that niceness is an inevitable outcome of any darwinian process—that would be a strawman. But the set of evolutionary pressures which gave rise to humans selected for individuals who were able to coexist in tribes, and this selection pressure produced, among other things, niceness as a general quality. At least for people we consider in our in-group.
It doesn’t even require an understanding of game theory. Non-psychopaths aren’t nice to others because they worked out the risk-reward calculations and determined niceness has the highest payoff. They’re nice because it’s nice to be nice, because that’s a feeling evolution selected for.
There’s no a priori reason why an AI can’t be evolved (*ahem*, “trained via reinforcement learning on human feedback”) to produce a similar drive for niceness.