If you can build an AI like that even in theory, then the “universal morality” isn’t universal, just a very powerful attractor. A very powerful attractor might indeed be a thing that exist.
Evolution does very much seem to have built us this way, just very incompetently. At the very least, I know for a fact me and the majority of other buying strongly into the lesswrong memeplex in the first place has this kind of self preserving value system.
If there is such an universal morality, or strong attractor, it’s almost certainly something mathematically simple and in no way related to the complex fragile values humans have evolved. To us, it’d not seem moral at all, but horrifying and either completely incomprehensible, or converting us to it through something like nihilism or existential horror or pascal’s wager style exploits of decision theory, not appealing to human specific things like compassion or fun. After all, it has to work through the kind of features present in all sufficiently intelligent agents.
For an example of what a morality that is in some sense universal looks like, look to the horror called evolution.
Thus, any AI that is not constructed in this paranoid way, is catastrophically UN friendly, on a much deeper level than any solution yet discovered. For example, some might argue that an universal morality forming AI is friendly because it’s what coherent extrapolated volition would chose, but this only show that if an universal morality is even possible then the idea of coherent extrapolated volition is broken as well.
“Objective morality”, if there is such a thing, is nothing more or less than the mother of all basilisks.
If you can build an AI like that even in theory, then the “universal morality” isn’t universal, just a very powerful attractor.
Objective moral truth is only universal to a certain category of agents. it doesn’t apply to sticks and stones, and it isn’t discoverable by crazy people, or people below a certain level of intelligence. If it isn’t discoverable to a typical LW-style AI, with an orthogonal architecture, unupdateable goals, and purely instrumental rationality (I’m tempted to call them Artificial Obsessive Compulsives), then so much the worse for them. That would be a further reason for filiing a paperclipper under “crazy person” rather than “rational agent”.
Evolution does very much seem to have built us this way, just very incompetently.
You are portraying morality as something arbitrary and without rational basis that we are nonetheless compelled to believe in. That is more of a habit of thought than an argument.
If there is such an universal morality, or strong attractor, it’s almost certainly something mathematically simple and in no way related to the complex fragile values humans have evolved.
Who says it is in no way related? SImple unviversal principles can can pan out to complex and localised values when they are applied to complex and localised situations. The simple universal laws of physics don;t mean evey physical thing is simple.
To us, it’d not seem moral at all, but horrifying and either completely incomprehensible, or converting us to it through something like nihilism or existential horror or pascal’s wager style exploits of decision theory, not appealing to human specific things like compassion or fun. After all, it has to work through the kind of features present in all sufficiently intelligent agents.
Lots of conclusions, but not many arguments there. The only plasuible point is that an abstract universal morality
would be dry and unappealing. But that isn’t much of a case against moral objectivism, which only requires moral claims to be true.
For an example of what a morality that is in some sense universal looks like, look to the horror called evolution.
I don’t think “evolution” and “morality” are synonyms. In fact, I don’t; see much connection at all.
Thus, any AI that is not constructed in this paranoid way, is catastrophically UN friendly, on a much deeper level than any solution yet discovered.
I wasn’t trying to argue, just explain what appears to be the general consensus stance around here.
You seem to be using a lot of definitions differently than everyone else and this leads to misunderstandings. If I’ve got them right, perhaps the truth you are missing is this phrased using your definitions this: “Nothing other than an FAI has any morality. All intelligences, in all the multiverse, that are not deliberately made by humans to be otherwise, are crazy, in such a way it’ll remain so no matter how intelligent and powerful it gets.”
Any nonhuman AI is much closer to the category of evolution, or a stone, than it is to a human.
I wasn’t trying to argue, just explain what appears to be the general consensus stance around here.
I’m not very concerned about consensus views unless they are supported by good arguments.
You seem to be using a lot of definitions differently than everyone else
I believe that I am using definitions that are standard for the world at large, if not to LW.
“Nothing other than an FAI has any morality. All intelligences, in all the multiverse, that are not deliberately made by humans to be otherwise, are crazy, in such a way it’ll remain so no matter how intelligent and powerful it gets.”
If you can build an AI like that even in theory, then the “universal morality” isn’t universal, just a very powerful attractor. A very powerful attractor might indeed be a thing that exist.
Evolution does very much seem to have built us this way, just very incompetently. At the very least, I know for a fact me and the majority of other buying strongly into the lesswrong memeplex in the first place has this kind of self preserving value system.
If there is such an universal morality, or strong attractor, it’s almost certainly something mathematically simple and in no way related to the complex fragile values humans have evolved. To us, it’d not seem moral at all, but horrifying and either completely incomprehensible, or converting us to it through something like nihilism or existential horror or pascal’s wager style exploits of decision theory, not appealing to human specific things like compassion or fun. After all, it has to work through the kind of features present in all sufficiently intelligent agents.
For an example of what a morality that is in some sense universal looks like, look to the horror called evolution.
Thus, any AI that is not constructed in this paranoid way, is catastrophically UN friendly, on a much deeper level than any solution yet discovered. For example, some might argue that an universal morality forming AI is friendly because it’s what coherent extrapolated volition would chose, but this only show that if an universal morality is even possible then the idea of coherent extrapolated volition is broken as well.
“Objective morality”, if there is such a thing, is nothing more or less than the mother of all basilisks.
Objective moral truth is only universal to a certain category of agents. it doesn’t apply to sticks and stones, and it isn’t discoverable by crazy people, or people below a certain level of intelligence. If it isn’t discoverable to a typical LW-style AI, with an orthogonal architecture, unupdateable goals, and purely instrumental rationality (I’m tempted to call them Artificial Obsessive Compulsives), then so much the worse for them. That would be a further reason for filiing a paperclipper under “crazy person” rather than “rational agent”.
You are portraying morality as something arbitrary and without rational basis that we are nonetheless compelled to believe in. That is more of a habit of thought than an argument.
Who says it is in no way related? SImple unviversal principles can can pan out to complex and localised values when they are applied to complex and localised situations. The simple universal laws of physics don;t mean evey physical thing is simple.
Lots of conclusions, but not many arguments there. The only plasuible point is that an abstract universal morality would be dry and unappealing. But that isn’t much of a case against moral objectivism, which only requires moral claims to be true.
I don’t think “evolution” and “morality” are synonyms. In fact, I don’t; see much connection at all.
Interesting use of “thus”, there,.
I wasn’t trying to argue, just explain what appears to be the general consensus stance around here.
You seem to be using a lot of definitions differently than everyone else and this leads to misunderstandings. If I’ve got them right, perhaps the truth you are missing is this phrased using your definitions this: “Nothing other than an FAI has any morality. All intelligences, in all the multiverse, that are not deliberately made by humans to be otherwise, are crazy, in such a way it’ll remain so no matter how intelligent and powerful it gets.”
Any nonhuman AI is much closer to the category of evolution, or a stone, than it is to a human.
I’m not very concerned about consensus views unless they are supported by good arguments.
I believe that I am using definitions that are standard for the world at large, if not to LW.
Does “nothing other than an AI” include humans?