I’m not sure I’m totally folllowing, but briefly FWIW:
This reminds me of an idea for a post, “Friendliness is probably a fairly natural abstraction, but Human is probably not”.
I’d call your goodness either niceness or friendliness. I’d define it as the tendency to want to help beings with desires to achieve those desires. I think the source in the natural world is the game-theoretic utility of cooperating for humans and other mammels (among other hypothetical life forms). This utility gives evolution an inclination to create mechanisms that make us good, nice, or friendly.
I don’t think this in itself helps hugely with alignment, except to note that there’s probably not a useful eniversal ethics that AGI will discover itself. Even though the above could be taken as a universal ethics of sorts, we humans won’t like the results; lots of beings beside us have desires, so a maximally friendly being will probably end up optimizing us away in favor of something that fits its criteria better. For more on how we’d probably lose a universal weighing of being good to sentient beings, to either insects below us or ASIs of some sort above us, see Roger Dearnaley’s 5. Moral Value for Sentient Animals? Alas, Not Yet and the remainder of that sequence.
The argument is that we probably need some sort of human chauvinism if we want human survival, and unfortunately I find this argument compelling. If there were a universal ethics, we probably wouldn’t much like it. And there’s just probably not. The game-theoretic virtue of cooperation applies to humans and many creatures, but not to all equally; a being that can improve its capabilities and make copies of itself would seem to have little need of cooperation.
Switching directions: Your claim that all intelligence is social intelligence seems wrong. Humans are trained by other humans, but we can accomplish a lot alone. The worry is that we get the same thing with an AGI, and it lacks both the cognitive weaknesses and arbitrary inlaid drives that make humans largely social creatures.
Your claim that all intelligence is social intelligence seems wrong. Humans are trained by other humans, but we can accomplish a lot alone.
Hm, would it help if I clarified that individual human minds have multiple internal parts too? So even when “alone” humans are still social by this definition.
I’m not sure I’m totally folllowing, but briefly FWIW:
This reminds me of an idea for a post, “Friendliness is probably a fairly natural abstraction, but Human is probably not”.
I’d call your goodness either niceness or friendliness. I’d define it as the tendency to want to help beings with desires to achieve those desires. I think the source in the natural world is the game-theoretic utility of cooperating for humans and other mammels (among other hypothetical life forms). This utility gives evolution an inclination to create mechanisms that make us good, nice, or friendly.
I don’t think this in itself helps hugely with alignment, except to note that there’s probably not a useful eniversal ethics that AGI will discover itself. Even though the above could be taken as a universal ethics of sorts, we humans won’t like the results; lots of beings beside us have desires, so a maximally friendly being will probably end up optimizing us away in favor of something that fits its criteria better. For more on how we’d probably lose a universal weighing of being good to sentient beings, to either insects below us or ASIs of some sort above us, see Roger Dearnaley’s 5. Moral Value for Sentient Animals? Alas, Not Yet and the remainder of that sequence.
The argument is that we probably need some sort of human chauvinism if we want human survival, and unfortunately I find this argument compelling. If there were a universal ethics, we probably wouldn’t much like it. And there’s just probably not. The game-theoretic virtue of cooperation applies to humans and many creatures, but not to all equally; a being that can improve its capabilities and make copies of itself would seem to have little need of cooperation.
Switching directions: Your claim that all intelligence is social intelligence seems wrong. Humans are trained by other humans, but we can accomplish a lot alone. The worry is that we get the same thing with an AGI, and it lacks both the cognitive weaknesses and arbitrary inlaid drives that make humans largely social creatures.
Thanks for commenting.
Hm, would it help if I clarified that individual human minds have multiple internal parts too? So even when “alone” humans are still social by this definition.