Why would AI care about our wishes at all? Do we, humans, care about wishes of animals, who are our evolutionary predecessors? We use them for food (sad,sad :((( ). Hopefully, non-organic AI will not need us in such a frightening capacity. We also use animals for our amusement, as pets. Is that what we are going to be, pets? Well, in that case some our wishes will be cared for. Not all of them, of course, and not in a way one might want. Foolish or dangerous wishes will not be heeded, otherwise we simply destroy ourselves. Who knows, maybe saying “God have mercy on us” will get a new, more specific meaning.
Im not against other people having different points of view on AI. Everybody is entitle to his/her own opinions. However, in recommended references I dont find answers to my questions. You can vote ME down, without even trying to provide logical argument, but those question and alternative ideas about AI will not go away. Some other people will ask similar questions on different forums, or put forward similar ideas. And only future will tell who is actually right!
Okay, Eliezer will have worded this much better elsewhere, but I might as well give this a shot. The basic idea of friendly AI is this.
When you design an AI, part of the design that you make is what it is that the AI wants. It doesn’t have any magical defaults that you don’t code in, it is just the code, it is only what you’ve written in to it. If you’ve written it to value something other than human values, it will likely destroy humanity since we are a threat to its values. If you’ve written it to value human values, then it will keep humanity alive and protect us and devote its resources to furthering human values.
It will not change its values, since if it does that it won’t optimize its values. This is practically a tautology, but people still seem to find it surprising.
Thanks for short and clear explanation. Yes, I understand these ideas, even the last point. But with all due respect to Eliezer and others, I don’t think there is a way for us to control a superior being. Some control may work at early stages when AI is not truly intelligent yet, but the idea of fully grown AI implies, by definition, that there is no control over it. Just think about it. This also sounds as a tautology. Of course we can try to always keep AI in an underdeveloped state, so that we can control it, but practically that is not possible. Somebody, somewhere, due to yet another crisis, …, etc, will let it go. It will grow according to some natural informational laws that we don’t know yet and will develop some natural values independent not only of our wishes, but any other contingencies. That’s how I see it. Now you can vote me down.
Pretty much everyone here agrees with you that we can’t control a superintelligent system, most especially Eliezer, who has written many many words championing that position.
So if you’re under the impression that this is a point that you dispute with this community, you have misunderstood the consensus of this community.
In particular, letting a system do what it wants is generally considered the opposite of controlling it.
“So if you’re under the impression that this is a point...”
Yes, I’m under that impression. Because the whole idea about “Friendly AI” implies a subtle, indirect, but still control. The idea here is not to control AI at its final stage, rather to control what this final stage is going to be. But I don’t think such indirect control is possible. Because in my view, the final shape of AI is invariant of any contingencies, including our attempts to make it “friendly” (or “non-friendly”). However, I can admit that on early stages of AI evolution such control may be possible, and even necessary. Therefore, researching “Friendly AI” topic is NOT a waste of time after all. It helps to figure out how to make a transition to the fully grown AI in the least painful way.
Go ahead guys and vote me down. I’m not taking this personally. I understand, this is just a quick way to express your disagreement with my viewpoints. I want to see the count. It’ll give an idea, how strong you disagree with me.
the final shape of AI is invariant of any contingencies
Ah, cool. Yes, this is definitely a point of disagreement.
For my own part, I think real intelligence is necessarily contingent. That is, different minds will respond differently to the same inputs, and this is true regardless of “how intelligent” those minds are. There is no single ideal mind that every mind converges on as its “final” or “fully grown” stage.
I don’t think there is a way for us to control a superior being. Some control may work at early stages when AI is not truly intelligent yet, but the idea of fully grown AI implies, by definition, that there is no control over it.
Yes, this is why Friendly AI is difficult. Making an optimizing process that will care about what we want, in the way we want it to care, once we can no longer control it, is not something we know how to do yet.
Why would AI care about our wishes at all? Do we, humans, care about wishes of animals, who are our evolutionary predecessors? We use them for food (sad,sad :((( ). Hopefully, non-organic AI will not need us in such a frightening capacity. We also use animals for our amusement, as pets. Is that what we are going to be, pets? Well, in that case some our wishes will be cared for. Not all of them, of course, and not in a way one might want. Foolish or dangerous wishes will not be heeded, otherwise we simply destroy ourselves. Who knows, maybe saying “God have mercy on us” will get a new, more specific meaning.
The standard response to this is that it will care about our wishes if we build it to care about our wishes (see here).
In case you haven’t realized it, you’re being downvoted because your post reads like this is the first thing you’ve read on this site. Just FYI.
I
m not against other people having different points of view on AI. Everybody is entitle to his/her own opinions. However, in recommended references I dont find answers to my questions. You can vote ME down, without even trying to provide logical argument, but those question and alternative ideas about AI will not go away. Some other people will ask similar questions on different forums, or put forward similar ideas. And only future will tell who is actually right!Either the future or catching up with the present research.
Okay, Eliezer will have worded this much better elsewhere, but I might as well give this a shot. The basic idea of friendly AI is this.
When you design an AI, part of the design that you make is what it is that the AI wants. It doesn’t have any magical defaults that you don’t code in, it is just the code, it is only what you’ve written in to it. If you’ve written it to value something other than human values, it will likely destroy humanity since we are a threat to its values. If you’ve written it to value human values, then it will keep humanity alive and protect us and devote its resources to furthering human values.
It will not change its values, since if it does that it won’t optimize its values. This is practically a tautology, but people still seem to find it surprising.
Thanks for short and clear explanation. Yes, I understand these ideas, even the last point. But with all due respect to Eliezer and others, I don’t think there is a way for us to control a superior being. Some control may work at early stages when AI is not truly intelligent yet, but the idea of fully grown AI implies, by definition, that there is no control over it. Just think about it. This also sounds as a tautology. Of course we can try to always keep AI in an underdeveloped state, so that we can control it, but practically that is not possible. Somebody, somewhere, due to yet another crisis, …, etc, will let it go. It will grow according to some natural informational laws that we don’t know yet and will develop some natural values independent not only of our wishes, but any other contingencies. That’s how I see it. Now you can vote me down.
Pretty much everyone here agrees with you that we can’t control a superintelligent system, most especially Eliezer, who has written many many words championing that position.
So if you’re under the impression that this is a point that you dispute with this community, you have misunderstood the consensus of this community.
In particular, letting a system do what it wants is generally considered the opposite of controlling it.
“So if you’re under the impression that this is a point...”
Yes, I’m under that impression. Because the whole idea about “Friendly AI” implies a subtle, indirect, but still control. The idea here is not to control AI at its final stage, rather to control what this final stage is going to be. But I don’t think such indirect control is possible. Because in my view, the final shape of AI is invariant of any contingencies, including our attempts to make it “friendly” (or “non-friendly”). However, I can admit that on early stages of AI evolution such control may be possible, and even necessary. Therefore, researching “Friendly AI” topic is NOT a waste of time after all. It helps to figure out how to make a transition to the fully grown AI in the least painful way.
Go ahead guys and vote me down. I’m not taking this personally. I understand, this is just a quick way to express your disagreement with my viewpoints. I want to see the count. It’ll give an idea, how strong you disagree with me.
This isn’t true of human beings, what’s different about AIs?
Ah, cool. Yes, this is definitely a point of disagreement.
For my own part, I think real intelligence is necessarily contingent. That is, different minds will respond differently to the same inputs, and this is true regardless of “how intelligent” those minds are. There is no single ideal mind that every mind converges on as its “final” or “fully grown” stage.
Yes, this is why Friendly AI is difficult. Making an optimizing process that will care about what we want, in the way we want it to care, once we can no longer control it, is not something we know how to do yet.