This is an old account. See https://www.lesswrong.com/users/joshuafox for my active account.
Joshua_Fox
Karma: 137
Why aren’t there dojos that teach rationality?
They’re called universities. My PhD adviser taught me to think rationally and caught many of the errors in my thinking, including those caused by biases. This eventually allowed me to catch many of my own errors.
No, it wasn’t perfect. It was limited to one academic area and was not explicitly aimed at teaching rationality and overcoming biases. (Then again, martial-arts senseis do not teach their skills “explicitly”, but by repetition and training.) Moreover, gaining subject-matter expertise took as much of the effort as learning rational thinking processes. But a PhD in a rigorous field from a good university is as close as you’ll find to a “color belt” (dare I say “black belt”?) in rationality.
What if, as we approach the Singularity, it is provably or near-provably necessary to do unethical things like killing a few people or letting them die to avoid the worst of Singularity outcomes?
(I am not referring here to whether we may create non-Friendly AGI. I am referring to scenarios even before the AGI “takes over.”)
Such scenarios seem not impossible, and creates ethical dilemmas along the lines of what Yudkowsky mentions here.