I don’t care if AI is Friendly or not.
[...]
I am mainly interested in that whatever AI we create does not paperclip the universe
You contradict yourself here. A Friendly AI is an intelligence which attempts to improve the well-being of humanity. A paperclip maximiser is an intelligence which does not, as it cares about something different and unrelated. Any sufficiently advanced AI is either one or the other or somewhere in between.
By “sufficiently advanced”, I mean an AI which is intelligent enough to consider the future of humanity and attempt to influence it.
You contradict yourself here. A Friendly AI is an intelligence which attempts to improve the well-being of humanity. A paperclip maximiser is an intelligence which does not, as it cares about something different and unrelated. Any sufficiently advanced AI is either one or the other or somewhere in between.
No; these are two types of AIs out of a larger design space.
You ignore, at the very least, the most important and most desirable case:
An AI that shares many of humanity’s values, and attempts to
achieve those values rather than increase the well-being of humanity.
A paperclip maximizer also has a dead-end goal—maximizing paperclips—which is what I object to. An non-Friendly AI just has no particular interest in humans; there is no other necessary claim you can make about it.
Even more hairy. Any primary goal will, I think, eventually end up with a paperclipper. We need more research into how intelligent beings (ie, humans) actually function. I do not think people, with rare exceptions, actually have primary goals, only temporary, contingent goals to meet temporary ends. That is one reason I don’t think much of utilitarianism—peoples “utilities” are almost always temporary, contingent, and self-limiting.
This is also one reason why I have said that I think provably Friendly AI is impossible. I will be glad to be proven wrong if it does turn out to be possible.
Ah, I see. I misunderstood your definition of “paperclip maximiser”; I assumed paperclip maximiser and Unfriendly AI were equivalent. Sorry.
Next question: if maximising paperclips or relentless self-optimisation is a dead-end goal, what is an example of a non-dead-end goal? Is there a clear border between the two, which will be obvious to the AI?
To my mind, if paperclip maximisation is a dead end, then so is everything else. The Second Law of Thermodynamics will catch up with you eventually. Nothing you create will endure forever. The only thing you can do is try to maximise your utility for as long as possible, and if that means paperclips, then so be it.
You contradict yourself here. A Friendly AI is an intelligence which attempts to improve the well-being of humanity. A paperclip maximiser is an intelligence which does not, as it cares about something different and unrelated. Any sufficiently advanced AI is either one or the other or somewhere in between.
By “sufficiently advanced”, I mean an AI which is intelligent enough to consider the future of humanity and attempt to influence it.
No; these are two types of AIs out of a larger design space. You ignore, at the very least, the most important and most desirable case: An AI that shares many of humanity’s values, and attempts to achieve those values rather than increase the well-being of humanity.
A paperclip maximizer also has a dead-end goal—maximizing paperclips—which is what I object to. An non-Friendly AI just has no particular interest in humans; there is no other necessary claim you can make about it.
You have merely redefined the goal from ‘the benefit of humanity’ to ‘non dead-end goal’, which may just be equally hairy.
Even more hairy. Any primary goal will, I think, eventually end up with a paperclipper. We need more research into how intelligent beings (ie, humans) actually function. I do not think people, with rare exceptions, actually have primary goals, only temporary, contingent goals to meet temporary ends. That is one reason I don’t think much of utilitarianism—peoples “utilities” are almost always temporary, contingent, and self-limiting.
This is also one reason why I have said that I think provably Friendly AI is impossible. I will be glad to be proven wrong if it does turn out to be possible.
Ah, I see. I misunderstood your definition of “paperclip maximiser”; I assumed paperclip maximiser and Unfriendly AI were equivalent. Sorry.
Next question: if maximising paperclips or relentless self-optimisation is a dead-end goal, what is an example of a non-dead-end goal? Is there a clear border between the two, which will be obvious to the AI?
To my mind, if paperclip maximisation is a dead end, then so is everything else. The Second Law of Thermodynamics will catch up with you eventually. Nothing you create will endure forever. The only thing you can do is try to maximise your utility for as long as possible, and if that means paperclips, then so be it.
What would be a non-friendly goal that isn’t dead-end? (N.B. Not a rhetorical question.)
Deciding that humanity was a poor choice for the dominant sapient species and should be replaced by (the improved descendants of) dolphins or octopi?