Most AI researchers don’t seem too concerned about friendliness. If I don’t know even close to as much about AI as they do, why should I be convinced by any argument that I know failed to convince them?
First, they might not be very unconvinced by the arguments. Videotext
Hugo de Garis was one of the speakers at the conference, and he polled the audience, asking: “If it were determined that the development of an artificial general intelligence would have a high likelihood of causing the extinction of the human race, how many of you feel that we should still proceed full speed ahead?” I looked around, expecting no one to raise their hand, and was shocked that half of the audience raised their hands. This says to me that we need a much greater awareness of morality among AI researchers.
Second, abstractly: it is much easier to see how things fail than how they succeed.
The argument that Friendliness is an important concern is an argument that GAIs systematically fail in certain ways.
For each GAI proposal, taboo “Friendly”. Think about what the Friendliness argument implies, and where it predicts the GAI would fail. Consider the designer’s response to the specific concern rather than to the whole Friendliness argument. If their response is that a patch would work, one can challenge that assertion as well if one understands a reason why the patch would fail. One doesn’t have to pit his or her (absent) technical understanding of Friendliness against a critic’s.
Ultimately my somewhat high belief that no present or foreseeable GAI design that ignores Frienliness would be safe for humanity is mostly a function of a few things: my trust in Omohundro/Eliezer plus my non-technical understanding plus my knowledge about several GAI designs that supposedly avoid the problem and I know don’t plus having heard bad arguments accepted as a refutation of Friendliness generally. It’s not based solely on trusting authority.
First, they might not be very unconvinced by the arguments. Video text
Second, abstractly: it is much easier to see how things fail than how they succeed.
The argument that Friendliness is an important concern is an argument that GAIs systematically fail in certain ways.
For each GAI proposal, taboo “Friendly”. Think about what the Friendliness argument implies, and where it predicts the GAI would fail. Consider the designer’s response to the specific concern rather than to the whole Friendliness argument. If their response is that a patch would work, one can challenge that assertion as well if one understands a reason why the patch would fail. One doesn’t have to pit his or her (absent) technical understanding of Friendliness against a critic’s.
Ultimately my somewhat high belief that no present or foreseeable GAI design that ignores Frienliness would be safe for humanity is mostly a function of a few things: my trust in Omohundro/Eliezer plus my non-technical understanding plus my knowledge about several GAI designs that supposedly avoid the problem and I know don’t plus having heard bad arguments accepted as a refutation of Friendliness generally. It’s not based solely on trusting authority.