The way you have explained this idea assumes a certain model of ethics/friendliness—that ethics is human value,and all human value indifferently. Other models make the problem a lot simpler.
It’s started already. Current technologies already share an ecosystem with humans and are being selected for some kind of friendliness.
It would probably be stymied by rapid takeoff, but so would all the alternatives....rapid takeoff towards ASI is the hard problem.
The way you have explained this idea assumes a certain model of ethics/friendliness—that ethics is human value,and all human value indifferently. Other models make the problem a lot simpler.
It’s started already. Current technologies already share an ecosystem with humans and are being selected for some kind of friendliness.
It would probably be stymied by rapid takeoff, but so would all the alternatives....rapid takeoff towards ASI is the hard problem.