Oddly, while I was at MIRI I thought the ontology identification problem was hard and absolutely critical, and it seemed Eliezer was more optimistic about it; he thought it would probably get solved along the way in AI capabilities development, because e.g. the idea of carbon atoms in diamond is a stable concept, and “you don’t forget how to ride a bike”. (Not sure if his opinion has changed)
I wonder if the difficulty is on a spectrum. It could be that he’s optimistic about explaining carbon atoms in diamond to an AGI in a stable way, but not the concept of kindness to humans. I’d certainly be more optimistic about the first.
Oddly, while I was at MIRI I thought the ontology identification problem was hard and absolutely critical, and it seemed Eliezer was more optimistic about it; he thought it would probably get solved along the way in AI capabilities development, because e.g. the idea of carbon atoms in diamond is a stable concept, and “you don’t forget how to ride a bike”. (Not sure if his opinion has changed)
I wonder if the difficulty is on a spectrum. It could be that he’s optimistic about explaining carbon atoms in diamond to an AGI in a stable way, but not the concept of kindness to humans. I’d certainly be more optimistic about the first.