Separate ‘discourse about the subject of friendly AI’, i.e. loose speculations about the future, the universe and everything else from ‘the project of friendly AI’, i.e. people trying to do actual math. Of the six assertions you make here only the one about acausal stuff seems relevant to actual research. The others are be important in deciding whether friendly AI is worth pursuing at all but if you start with the assumption that friendly AI matters, they don’t seem to matter much.
The assumptions that I criticize may be providing a lot of the motivation, but you can think that e.g. solving the problem of ethical stability under self-modification is important, without believing that stuff; and meanwhile, a lot of people must be encountering the concept of Friendly AI as part of a package which includes the futurist maximalism and the peculiar metaphysics. I suppose I’m saying that Friendly AI needs to be saved from the subculture that most vigorously supports it, because it contains ideas that really are important.
How about we plant seeds for a new culture intended to design mechanism-designing mechanisms? One seed institution will be this comment exchange. We can bootstrap from there.
Separate ‘discourse about the subject of friendly AI’, i.e. loose speculations about the future, the universe and everything else from ‘the project of friendly AI’, i.e. people trying to do actual math. Of the six assertions you make here only the one about acausal stuff seems relevant to actual research. The others are be important in deciding whether friendly AI is worth pursuing at all but if you start with the assumption that friendly AI matters, they don’t seem to matter much.
The assumptions that I criticize may be providing a lot of the motivation, but you can think that e.g. solving the problem of ethical stability under self-modification is important, without believing that stuff; and meanwhile, a lot of people must be encountering the concept of Friendly AI as part of a package which includes the futurist maximalism and the peculiar metaphysics. I suppose I’m saying that Friendly AI needs to be saved from the subculture that most vigorously supports it, because it contains ideas that really are important.
How about we plant seeds for a new culture intended to design mechanism-designing mechanisms? One seed institution will be this comment exchange. We can bootstrap from there.