However, some people think that this can be reduced to saying that we should just make AIs try to make people smile… which could result in anything from world-wide happiness drugs to surgically altering our faces into permanent smiles to making lots of tiny models of perfectly-smiling humans.
But you implicitly assume that it is given the incentive to develop the cognitive flexibility and comprehension to act in a real-world environment and do those things but at the same time you propose that the same people who are capable of giving it such extensive urges fail on another goal in such a blatant and obvious way. How does that make sense?
See the previous articles here about memetic immunity: when you teach hunter-gatherer tribes about Christianity, they interrpret the bible literally and do all sorts of things that “real” Christians don’t. An AI isn’t going to be smart enough to not take you seriously when you tell it that...
The difference between the hunter-gatherer and the AI is that the hunter-gatherer already posses a wide range of conceptual frameworks and incentives. An AI isn’t going to do something without someone to carefully and deliberately telling it do do so and what to do. It won’t just read the Bible and come to the conclusion that it should convert all humans to Christianity. Where would such an incentive come from?
You don’t need to be very creative or smart to come up with LOTS of ways for this command sequence to have bugs with horrible consequences, if the AI has any ability to influence the world.
The AI is certainly very creative and smart if it can influence the world dramatically. You allow it to be that smart, you allow it to care to do so, but you don’t allow it to comprehend what you actually mean? What I’m trying to pinpoint here is that you seem to believe that there are many pathways that lead to superhuman abilities yet all of them fail to comprehend some goals while still being able to self-improve on them.
you implicitly assume that it is given the incentive to develop the cognitive flexibility and comprehension to act in a real-world environment and do those things but at the same time you propose that the same people who are capable of giving it such extensive urges fail on another goal in such a blatant and obvious way. How does that make sense?
Because people make stupid mistakes, especially when programming. And telling your fully-programmed AI what you want it to do still counts as programming.
At this point, I am going to stop my reply, because the remainder of your comment consists of taking things I said out of context and turning them into irrelevancies:
I didn’t say an AI would try to convert people to Christianity—I said that humans without sufficient shared background will interpret things literally, and so would AIs.
I didn’t say the AI needed to be creative or smart, I said you wouldn’t need to be creative or smart to make a list of ways those three simple instructions could be given a disastrous literal interpretation.
you seem to believe that there are many pathways that lead to superhuman abilities yet all of them fail to comprehend some goals while still being able to self-improve on them.
There are many paths to superhuman ability, as humans really aren’t that smart.
This also means that you can easily be superhuman in ability, and still really dumb—in terms of comprehending what humans mean… but don’t actually say.
But you implicitly assume that it is given the incentive to develop the cognitive flexibility and comprehension to act in a real-world environment and do those things but at the same time you propose that the same people who are capable of giving it such extensive urges fail on another goal in such a blatant and obvious way. How does that make sense?
The difference between the hunter-gatherer and the AI is that the hunter-gatherer already posses a wide range of conceptual frameworks and incentives. An AI isn’t going to do something without someone to carefully and deliberately telling it do do so and what to do. It won’t just read the Bible and come to the conclusion that it should convert all humans to Christianity. Where would such an incentive come from?
The AI is certainly very creative and smart if it can influence the world dramatically. You allow it to be that smart, you allow it to care to do so, but you don’t allow it to comprehend what you actually mean? What I’m trying to pinpoint here is that you seem to believe that there are many pathways that lead to superhuman abilities yet all of them fail to comprehend some goals while still being able to self-improve on them.
Because people make stupid mistakes, especially when programming. And telling your fully-programmed AI what you want it to do still counts as programming.
At this point, I am going to stop my reply, because the remainder of your comment consists of taking things I said out of context and turning them into irrelevancies:
I didn’t say an AI would try to convert people to Christianity—I said that humans without sufficient shared background will interpret things literally, and so would AIs.
I didn’t say the AI needed to be creative or smart, I said you wouldn’t need to be creative or smart to make a list of ways those three simple instructions could be given a disastrous literal interpretation.
There are many paths to superhuman ability, as humans really aren’t that smart.
This also means that you can easily be superhuman in ability, and still really dumb—in terms of comprehending what humans mean… but don’t actually say.