Well my idea is not that creative, or even new, meaning that even if I hadn’t just posted it online an AI could still have conceivably read it somewhere else, and I do think creativity is a property of any sufficiently general intelligence that we might create, but those points are secondary.
No one here will argue that an unFriendly AI will do “bad things” because it doesn’t care (about what?). It will do bad things because it cares more about something else. Nor is “bad” an absolute: actions may be bad for some people and not for others, and there are moral systems under which actions can be firmly called “wrong”, but where all alternative actions are also “wrong”. Problems like that arise even for humans; in an AI the effects could be very ugly indeed.
And to clarify, I expect any AI that isn’t completely ignorant, let alone general, to know that we don’t like to be turned into broccoli. My example was of changing what humans want. Wireheading is the obvious candidate of a desire that an AI might want to implant.
What I meant is that the argument is that you have to make it care about humans so as not to harm them. Yet it is assumed that it does a lot without having to care about it, e.g. creating paperclips or self-improvement. My question is, why do people believe that you don’t have to make it care to do those things but you have to make it care to not harm humans. It is clear that if it only cares about one thing, doing that one thing could harm humans. Yet why would it do that one thing to an extent that is either not defined or which it is not deliberately made to care about. The assumptions seems to be that AI’s will do something, anything but being passive. Why isn’t limited behavior, failure and impassivity together not more likely than harming humans as a result of own goals or as a result to follow all goals but the one that limits its scope?
Well my idea is not that creative, or even new, meaning that even if I hadn’t just posted it online an AI could still have conceivably read it somewhere else, and I do think creativity is a property of any sufficiently general intelligence that we might create, but those points are secondary.
No one here will argue that an unFriendly AI will do “bad things” because it doesn’t care (about what?). It will do bad things because it cares more about something else. Nor is “bad” an absolute: actions may be bad for some people and not for others, and there are moral systems under which actions can be firmly called “wrong”, but where all alternative actions are also “wrong”. Problems like that arise even for humans; in an AI the effects could be very ugly indeed.
And to clarify, I expect any AI that isn’t completely ignorant, let alone general, to know that we don’t like to be turned into broccoli. My example was of changing what humans want. Wireheading is the obvious candidate of a desire that an AI might want to implant.
What I meant is that the argument is that you have to make it care about humans so as not to harm them. Yet it is assumed that it does a lot without having to care about it, e.g. creating paperclips or self-improvement. My question is, why do people believe that you don’t have to make it care to do those things but you have to make it care to not harm humans. It is clear that if it only cares about one thing, doing that one thing could harm humans. Yet why would it do that one thing to an extent that is either not defined or which it is not deliberately made to care about. The assumptions seems to be that AI’s will do something, anything but being passive. Why isn’t limited behavior, failure and impassivity together not more likely than harming humans as a result of own goals or as a result to follow all goals but the one that limits its scope?