Aron:If we take for granted that strong AI is so fricking hard we can’t get there in one step, we have to start looking at what steps we can take today that are productive.
Well we probably want to work on friendly goal systems rather than how to get AGI to work. Ideally, you want to know exactly what kind of motivational system your AGI should have before you (or anyone else) knows how to build an AGI.
My personal estimate is veering towards “AGI won’t come first”, because a number of clever people like Robin Hanson and the guys at the Future of Humanity Institute think whole brain emulation will come first, and have good arguments for that conclusion. However, we should be ready for the contingency that AGI really gets going, which is why I think that Eliezer & SingInst are doing such a valuable job.
So, I would modify Aron’s request to: If we take for granted that Friendly AI is so hard we can’t get there in one step, we have to start looking at what steps we can take today that are productive. What productive steps towards FAI can we take today?
Aron:If we take for granted that strong AI is so fricking hard we can’t get there in one step, we have to start looking at what steps we can take today that are productive.
Well we probably want to work on friendly goal systems rather than how to get AGI to work. Ideally, you want to know exactly what kind of motivational system your AGI should have before you (or anyone else) knows how to build an AGI.
My personal estimate is veering towards “AGI won’t come first”, because a number of clever people like Robin Hanson and the guys at the Future of Humanity Institute think whole brain emulation will come first, and have good arguments for that conclusion. However, we should be ready for the contingency that AGI really gets going, which is why I think that Eliezer & SingInst are doing such a valuable job.
So, I would modify Aron’s request to: If we take for granted that Friendly AI is so hard we can’t get there in one step, we have to start looking at what steps we can take today that are productive. What productive steps towards FAI can we take today?