Especially for people intending to program friendly AI, who need to understand the needs of other people (although I doubt very much AI will be developed or that MIRI will ever really start coding it. Plus I do not want it to exist. But it is just me.)
The plan to write an AI that will implement the Coherent Extrapolated Volition of all of humanity doesn’t involve talking to any of the affected humans. The plan is, literally, to first build an earlier AI that will do the interacting with all those other people for them.
That link only explains the concept of CEV as one possible idea related to building FAI, and a problematic one at that. But you’re making it sound like CEV being the only possible approach was an opinion that had already been set in stone.
AFAIK, it’s one idea that’s being considered, but I don’t think there’s currently enough confidence in any particular approach to call it The Plan. “The Plan” is more along the lines of “let’s experiment with a lot of approaches and see which ones seem the most promising”; the most recent direction that that plan has produced is a focus on general FAI math research, which may or may not eventually lead to something CEV-like.
The plan to write an AI that will implement the Coherent Extrapolated Volition of all of humanity doesn’t involve talking to any of the affected humans. The plan is, literally, to first build an earlier AI that will do the interacting with all those other people for them.
That link only explains the concept of CEV as one possible idea related to building FAI, and a problematic one at that. But you’re making it sound like CEV being the only possible approach was an opinion that had already been set in stone.
As far as I understood, it was still the plan as of quite recently (last coupla years). Has this changed?
AFAIK, it’s one idea that’s being considered, but I don’t think there’s currently enough confidence in any particular approach to call it The Plan. “The Plan” is more along the lines of “let’s experiment with a lot of approaches and see which ones seem the most promising”; the most recent direction that that plan has produced is a focus on general FAI math research, which may or may not eventually lead to something CEV-like.