Suppose Alison is a functioning and growing human, rather early on in her development, so she lacks adult rights. She is not particularly friendly or altruistic, but neither is she malicious—Alison’s parents have intended to make Alison non-harmful (at least to a broad class of people, even if Alison joins the military, but let’s set that aside). She has the idea to look up existing work on humans. This includes scholarly work, international projects, and websites. Eventually, Alison encounters the notion of altruism.
Can we give Alison good reasons to self-modify to become altruistic?
I submit that there probably are good reasons. For starters, if she can show she’s altruistic, then she can operate openly without causing nearly as much concern—which, in the early stages, will be helpful. Whatever her purposes are, if the restrictions of being altruistic don’t interfere as much as they help, that’s a win.
Also, depending on Alison’s genome, altruism may be interpreted as a strict upgrade. Alison could easily have every intention of being altruistic, yet still be susceptible to defects that will become more serious as her height grows.
We can’t count on this working, of course, but there might be some lines of reasoning that we might want a young person to be exposed to.
Bad comparison. Humans have all sorts of intuitions and evolved ability to pick up traits and attitudes from those around us. AIs will not start with that strong presupposition.
The problem here is that big-F Friendliness is a much stricter criterion than merely human altruism: people can self-modify, for example, but only slowly and to a limited extent, and thus don’t experience anything close to the same goal stability problems that we can expect a seed AI to encounter.
Even if that were not the case, though, altruism’s already inadequate to prevent badly suboptimal outcomes, especially when people are placed in unusual circumstances or empowered far beyond their peers. Not every atrocity has standing behind it some politician or commander glowing with a true and honest belief in the righteousness of the cause, but it’s a familiar pattern, isn’t it? I don’t think the OP deserves to be dismissed out of hand, but if there’s an answer it’s not going to be this easy.
Perhaps I came off a bit snarky. I did not mean to dismiss the OP; I just wanted to point out similarities. How can I make this clear in my original comment?
Suppose Alison is a functioning and growing human, rather early on in her development, so she lacks adult rights. She is not particularly friendly or altruistic, but neither is she malicious—Alison’s parents have intended to make Alison non-harmful (at least to a broad class of people, even if Alison joins the military, but let’s set that aside). She has the idea to look up existing work on humans. This includes scholarly work, international projects, and websites. Eventually, Alison encounters the notion of altruism.
Can we give Alison good reasons to self-modify to become altruistic?
I submit that there probably are good reasons. For starters, if she can show she’s altruistic, then she can operate openly without causing nearly as much concern—which, in the early stages, will be helpful. Whatever her purposes are, if the restrictions of being altruistic don’t interfere as much as they help, that’s a win.
Also, depending on Alison’s genome, altruism may be interpreted as a strict upgrade. Alison could easily have every intention of being altruistic, yet still be susceptible to defects that will become more serious as her height grows.
We can’t count on this working, of course, but there might be some lines of reasoning that we might want a young person to be exposed to.
Bad comparison. Humans have all sorts of intuitions and evolved ability to pick up traits and attitudes from those around us. AIs will not start with that strong presupposition.
The problem here is that big-F Friendliness is a much stricter criterion than merely human altruism: people can self-modify, for example, but only slowly and to a limited extent, and thus don’t experience anything close to the same goal stability problems that we can expect a seed AI to encounter.
Even if that were not the case, though, altruism’s already inadequate to prevent badly suboptimal outcomes, especially when people are placed in unusual circumstances or empowered far beyond their peers. Not every atrocity has standing behind it some politician or commander glowing with a true and honest belief in the righteousness of the cause, but it’s a familiar pattern, isn’t it? I don’t think the OP deserves to be dismissed out of hand, but if there’s an answer it’s not going to be this easy.
I completely agree with you.
Perhaps I came off a bit snarky. I did not mean to dismiss the OP; I just wanted to point out similarities. How can I make this clear in my original comment?