I think the problem is much more profound than you suggest. It is not something that rationalists can simply take on with a non-infinitesimal confidence that progress will be made. Certainly not amateur rationalists doing philosophy in their spare time (not that this isn’t healthy). I don’t mean to say that rationalists should give up, but we have to choose how to act in the meantime.
Personally, I find the situation so desperate that I am prepared to simply assume moral realism when I am deciding how to act, with the knowledge that this assumption is very implausible. I don’t believe this makes me irrational. In fact, given our current understanding of the problem, I don’t know of any other reasonable approaches.
Incidentally, this position is reminiscent of both Pascal’s wager and of an attitude towards morality and AI which Eliezer claimed to previously hold but now rejects as flawed.
I think the problem is much more profound than you suggest. It is not something that rationalists can simply take on with a non-infinitesimal confidence that progress will be made. Certainly not amateur rationalists doing philosophy in their spare time (not that this isn’t healthy). I don’t mean to say that rationalists should give up, but we have to choose how to act in the meantime.
Personally, I find the situation so desperate that I am prepared to simply assume moral realism when I am deciding how to act, with the knowledge that this assumption is very implausible. I don’t believe this makes me irrational. In fact, given our current understanding of the problem, I don’t know of any other reasonable approaches.
Incidentally, this position is reminiscent of both Pascal’s wager and of an attitude towards morality and AI which Eliezer claimed to previously hold but now rejects as flawed.