So basically I mostly don’t think that Eliezer expects that there are natural abstractions, but that he thinks the problem is that we don’t have sufficient understanding of AGI cognition to robustly point goals of an AGI.
I think Eliezer thinks AGI cognition is quite different from human cognition, which makes it harder. But even if we were to develop brain-like AGI which worked sorta more like humans, the way this cognition looks like deep down still seems sorta alien, and we still don’t nearly have the skill to robustly point the goal of an AGI even if it was brain-like.
So basically I mostly don’t think that Eliezer expects that there are natural abstractions, but that he thinks the problem is that we don’t have sufficient understanding of AGI cognition to robustly point goals of an AGI.
I think Eliezer thinks AGI cognition is quite different from human cognition, which makes it harder. But even if we were to develop brain-like AGI which worked sorta more like humans, the way this cognition looks like deep down still seems sorta alien, and we still don’t nearly have the skill to robustly point the goal of an AGI even if it was brain-like.