(ZMDavis wrote:) “But AGI is [...] not being a judge or whoever writes laws.”
If Eliezer turns out to be right about the power of recursive self-improvement, then I wouldn’t be so sure.”
Argh. I didn’t mean that as a critique on EY’s prowess as an AGI theorist or programmer. I doubt Jesus would’ve wanted people to deify him, just to be nice to eachother. I doubt EY meant for his learning of philosophy to be interpreted as some sort of Moral code, he was just arrogant enough not to state he was sometimes using his list to as a tool to develop his own philosophy. I’m assuming any AGI project would be a team, and I’d doubt he’d challenge his best comparitive advantage is not ethics. Maybe he plans on writing the part of the code that tells an AGI how to stop using resources for a given job.
Another example of a real-world Moral quandry that the real world would love H+ disucssion lists to take on, is the issue of how much medical care to invest in end-of-life patients. Medical advances will continue to make more expensive treatment options available. In Winnipeg, there was a case recently where a patient in a terminal coma had his family insist on not taking him off life support. In Canada in the last decade or so, the decision was based on a doctor’s prescription. Now it also encompases family and the patient’s previous wishes. 3 doctors quit over the case. My first instinct was to suggest doctors be trained exclusively to be coma-experts, but it seems medical boards might already have accomplished this. I admire a fighting spirit, and one isolated case doesn’t tax the healthcare system much. But if this becomes a regular occurrence...this is another of many real-world examples that require intelligent thought. Subhan’s position has already been proven wrong many many times. There are cognitive biases but they aren’t nearly as strong or all-encompassing as is being suggested here. For example, I’d guess every reader on this list is aware that other people are capable of suffering and feeling happiness that corresponds with their own experiences. This isn’t mirror-neurons or some other “bias”, it is simple grade school deduction that refutes Subhan’s position. You don’t have to be highly Moral, to admit it’s out there in some people. For instance, most children get what Subhan doesn’t.