The intelligence of the agent is highly relevant, yes, but showing that AGI would have a major advantage in this area in particular would require showing that there are plausible ways of attacking the domain which humans are currently incapable of exploiting but which an AGI would be capable of exploiting.
Off the top of my head, I would expect that an AGI might have a larger relative advantage in something like social science, where hypothesis testing requires making sense of huge datasets and correctly interpreting complex statistical relationships and causal chains and integrating them with the researcher’s existing knowledge—something that humans are quite bad at, due to not having evolved for the task. (Though evolution has spent a long while optimizing humans to have an intuitive understanding of the motives of other humans, one which might take a long while for an AGI to catch up with—but then, that understanding can also be a disadvantage in evaluating correct but counter-intuitive hypotheses.) In contrast, something well-understood like physics seems like a much likelier candidate for a field where you can’t come up with any major intelligence enhancement technique that computer-aided humans couldn’t exploit equally well.
“The more reliable a field’s current predictions, the less likely an AGI is to have a relative advantage” seems like a potentially useful heuristic—having a field of science consistently make correct predictions is a sign of our evolved cognitive faculties already either being relatively good at it or at least effectively aided by computerized tools, suggesting that there is less room for improvement than in other fields. Physics is possibly the most reliable field of science there is, which would suggest that the biggest AGI advantages would lie in other fields.
Of course an AGI could still have a big advantage in physics due to general considerations such as thinking faster, instances of it coordinating better among themselves, etc., but those considerations would apply equally to all fields of science, not just physics. It doesn’t seem impossible that an AGI wouldn’t have any qualitative advantages over humans when it came to physics.
Yes, physics experiments are easier to interpret than social experiments, but (as you yourself point out), the current state of social science shows that this is also the case when humans are doing the experimenting.
No, I’m not talking about the interpretation of experiments so much as the risks while learning. People grow up (if at all fortunate) with the chance to do a lot of low-stakes attempts while dealing with other people. Even so, very few end up as skilled politicians.
If the AI needs to be able to navigate complex negotiations and signalling, it isn’t going to start with the benefit of a child’s slack for learning. If it needs to practice with people rather than simulations (I’m guessing it will), it could take years to build up to super-negotiator.
I could be wrong. Perhaps the AI can build on algorithms which aren’t obvious to people, and/or use improved sensory abilities, and/or be able to capitalize on a huge bank of information.
The Internet provides plenty of opportunities for anonymous interaction with people, perfect for running safe experiments. And the amounts of raw information that you could process before ever needing to run your own experiments is just enormous—here I’m not talking about the scientific papers, but all the forum threads, mailing list archives, chatlogs, etc. etc. that exist online. These not only demonstrate how online social interaction works, but also contain plenty of data where people report on and analyze their various real-life social interactions (“hey guys, the funniest thing happened today...”, “I was so creeped out on my ride home”, “I met the most charming person”). That is just an insane corpus of what works and what all the different failure modes are. I would expect that it would take one a long time before they could come up with hypotheses that weren’t in principle better answered by searching that corpus than doing your own experiments.
Of course, that is highly unstructured data, and it might be quite hard to effectively categorize it in a way that allows for efficient searches. So doing your own experiments might still be more effective. But again, plenty of anonymous forums exist for that.
The intelligence of the agent is highly relevant, yes, but showing that AGI would have a major advantage in this area in particular would require showing that there are plausible ways of attacking the domain which humans are currently incapable of exploiting but which an AGI would be capable of exploiting.
Off the top of my head, I would expect that an AGI might have a larger relative advantage in something like social science, where hypothesis testing requires making sense of huge datasets and correctly interpreting complex statistical relationships and causal chains and integrating them with the researcher’s existing knowledge—something that humans are quite bad at, due to not having evolved for the task. (Though evolution has spent a long while optimizing humans to have an intuitive understanding of the motives of other humans, one which might take a long while for an AGI to catch up with—but then, that understanding can also be a disadvantage in evaluating correct but counter-intuitive hypotheses.) In contrast, something well-understood like physics seems like a much likelier candidate for a field where you can’t come up with any major intelligence enhancement technique that computer-aided humans couldn’t exploit equally well.
“The more reliable a field’s current predictions, the less likely an AGI is to have a relative advantage” seems like a potentially useful heuristic—having a field of science consistently make correct predictions is a sign of our evolved cognitive faculties already either being relatively good at it or at least effectively aided by computerized tools, suggesting that there is less room for improvement than in other fields. Physics is possibly the most reliable field of science there is, which would suggest that the biggest AGI advantages would lie in other fields.
Of course an AGI could still have a big advantage in physics due to general considerations such as thinking faster, instances of it coordinating better among themselves, etc., but those considerations would apply equally to all fields of science, not just physics. It doesn’t seem impossible that an AGI wouldn’t have any qualitative advantages over humans when it came to physics.
People might have a temporary but significant advantage in avoiding disastrous mistakes in dealing with other people.
Building from experiments in MNT has got to be easier for an AI than building from experiments with people.
I’m not even sure that social science covers how to not make bad mistakes in fraught political situations.
Yes, physics experiments are easier to interpret than social experiments, but (as you yourself point out), the current state of social science shows that this is also the case when humans are doing the experimenting.
No, I’m not talking about the interpretation of experiments so much as the risks while learning. People grow up (if at all fortunate) with the chance to do a lot of low-stakes attempts while dealing with other people. Even so, very few end up as skilled politicians.
If the AI needs to be able to navigate complex negotiations and signalling, it isn’t going to start with the benefit of a child’s slack for learning. If it needs to practice with people rather than simulations (I’m guessing it will), it could take years to build up to super-negotiator.
I could be wrong. Perhaps the AI can build on algorithms which aren’t obvious to people, and/or use improved sensory abilities, and/or be able to capitalize on a huge bank of information.
The Internet provides plenty of opportunities for anonymous interaction with people, perfect for running safe experiments. And the amounts of raw information that you could process before ever needing to run your own experiments is just enormous—here I’m not talking about the scientific papers, but all the forum threads, mailing list archives, chatlogs, etc. etc. that exist online. These not only demonstrate how online social interaction works, but also contain plenty of data where people report on and analyze their various real-life social interactions (“hey guys, the funniest thing happened today...”, “I was so creeped out on my ride home”, “I met the most charming person”). That is just an insane corpus of what works and what all the different failure modes are. I would expect that it would take one a long time before they could come up with hypotheses that weren’t in principle better answered by searching that corpus than doing your own experiments.
Of course, that is highly unstructured data, and it might be quite hard to effectively categorize it in a way that allows for efficient searches. So doing your own experiments might still be more effective. But again, plenty of anonymous forums exist for that.
… It’s programming and math. Humans suck at programming and math.