The assumptions that I criticize may be providing a lot of the motivation, but you can think that e.g. solving the problem of ethical stability under self-modification is important, without believing that stuff; and meanwhile, a lot of people must be encountering the concept of Friendly AI as part of a package which includes the futurist maximalism and the peculiar metaphysics. I suppose I’m saying that Friendly AI needs to be saved from the subculture that most vigorously supports it, because it contains ideas that really are important.
How about we plant seeds for a new culture intended to design mechanism-designing mechanisms? One seed institution will be this comment exchange. We can bootstrap from there.
The assumptions that I criticize may be providing a lot of the motivation, but you can think that e.g. solving the problem of ethical stability under self-modification is important, without believing that stuff; and meanwhile, a lot of people must be encountering the concept of Friendly AI as part of a package which includes the futurist maximalism and the peculiar metaphysics. I suppose I’m saying that Friendly AI needs to be saved from the subculture that most vigorously supports it, because it contains ideas that really are important.
How about we plant seeds for a new culture intended to design mechanism-designing mechanisms? One seed institution will be this comment exchange. We can bootstrap from there.