@Richard Hollerith: “I realize that it is foolish to create a seed AI with the intention that it will figure out morality after it is launched: the creators cannot escape the need to make a real moral choice.”
I’m actually not so sure. We humans are profoundly confused about morality, and I find it hard to see anything definate through this confusion. I currently suspect that a number of category errors are at work in contepmorary thought surrounding “ethics” and “values”. It especially worries me that a lot of people in the singularitarian/transhumanist movement are utilitarians. Surely we should have moved beyond that by now!
@Richard Hollerith: “I realize that it is foolish to create a seed AI with the intention that it will figure out morality after it is launched: the creators cannot escape the need to make a real moral choice.”
I’m actually not so sure. We humans are profoundly confused about morality, and I find it hard to see anything definate through this confusion. I currently suspect that a number of category errors are at work in contepmorary thought surrounding “ethics” and “values”. It especially worries me that a lot of people in the singularitarian/transhumanist movement are utilitarians. Surely we should have moved beyond that by now!