“But I’m skeptical that this uniformity extends to system II. The system II abilities of the best rationalists of today may depend significantly on their having learned a set of reasoning skills developed by their culture over a long period of time.”
That’s precisely the point; the biological difference between humans is not that great, so the huge differences we see in human accomplishment must be due in large part to other factors.
Agreed. But I think that if you put up a scale of “intelligence” then people will take into account abilities other than those that are (in some ill-defined and crude sense) dependent only on biology. And if we’re talking about building an AI, then I’m not sure how useful it is to attempt to distinguish biological and other factors. If Feynman or Von Neumann or Pauling really are a bigger distance from the VI than Eliezer allows in terms of their system II thinking ability, then that seems to me significant independently of whether that distance is best explained in terms of the interaction of powerful learned thinking tools (scientific method, analogical reasoning, logical argument) with high g factor rather than the g factor on its own.
“But I’m skeptical that this uniformity extends to system II. The system II abilities of the best rationalists of today may depend significantly on their having learned a set of reasoning skills developed by their culture over a long period of time.”
That’s precisely the point; the biological difference between humans is not that great, so the huge differences we see in human accomplishment must be due in large part to other factors.
Agreed. But I think that if you put up a scale of “intelligence” then people will take into account abilities other than those that are (in some ill-defined and crude sense) dependent only on biology. And if we’re talking about building an AI, then I’m not sure how useful it is to attempt to distinguish biological and other factors. If Feynman or Von Neumann or Pauling really are a bigger distance from the VI than Eliezer allows in terms of their system II thinking ability, then that seems to me significant independently of whether that distance is best explained in terms of the interaction of powerful learned thinking tools (scientific method, analogical reasoning, logical argument) with high g factor rather than the g factor on its own.