“ME, morals can be derived from game theory… ”—I disagree. Game theory doesn’t tell you what you should do, it only tells you how to do it. - That was almost what I intended to say, but I somehow failed to formulate it well so you understood I had said the contrary...
Of course, what you have said isn’t sufficiently precise to be either correct or incorrect—words like “murder”, “intelligent” are very much in need of defining precisely. - I’m not sure that defining precisely what is murder is important for this debate. Obviously you can make the chains of definitions as long as you wish, but somewhere you have to stop and consider the words “primary” with “intuitive meaning”. If you think murder is too ambiguous, imagine something else which most people find wrong, the arguments remain the same.
Laws exist so that society functions correctly. - What does mean “correctly” in this statement?
An AI that randomly murdered people would not benefit from having those people around, so it would not be as intelligent/successful as a similar system which didn’t murder. - How can you know what would constitute a “benefit” for the AI? Most species on Earth would benefit (in the evolutionary sense) from human extinction, why not an AI?
Roko,
“ME, morals can be derived from game theory… ”—I disagree. Game theory doesn’t tell you what you should do, it only tells you how to do it. - That was almost what I intended to say, but I somehow failed to formulate it well so you understood I had said the contrary...
Of course, what you have said isn’t sufficiently precise to be either correct or incorrect—words like “murder”, “intelligent” are very much in need of defining precisely. - I’m not sure that defining precisely what is murder is important for this debate. Obviously you can make the chains of definitions as long as you wish, but somewhere you have to stop and consider the words “primary” with “intuitive meaning”. If you think murder is too ambiguous, imagine something else which most people find wrong, the arguments remain the same.
Laws exist so that society functions correctly. - What does mean “correctly” in this statement?
An AI that randomly murdered people would not benefit from having those people around, so it would not be as intelligent/successful as a similar system which didn’t murder. - How can you know what would constitute a “benefit” for the AI? Most species on Earth would benefit (in the evolutionary sense) from human extinction, why not an AI?