I agree with you on both counts—that most human cognition is simpler than it appears in particular. But some of it isn’t, and that’s probably the really critical part when we talk about strong AI.
For instance, I think that a computer could write a “Turing Novel” that would be indistinguishable from some human-made fiction with just a little bit of human editing, and that would still leave us quite far from FOOMable AI (I don’t mean this could happen today, but say in 10 years).
I agree with you on both counts—that most human cognition is simpler than it appears in particular. But some of it isn’t, and that’s probably the really critical part when we talk about strong AI.
For instance, I think that a computer could write a “Turing Novel” that would be indistinguishable from some human-made fiction with just a little bit of human editing, and that would still leave us quite far from FOOMable AI (I don’t mean this could happen today, but say in 10 years).