Agency (proficient tool use with high reliability for costly actions) might be sufficient to maintain an RSI loop (as an engineer, using settled methodology) even while lacking crucial capabilities (such as coming up with important novel ideas), eventually developing those capabilities without any human input. But even if it works like this, the AI speed advantage might be negated by lack of those capabilities, so that human-led AI research is still faster and mostly bottlenecked by availability and cost of compute.
This is a bit of a separate question, but it’s one I’m very interested in. I think the advantages of general problem-solving abilities will be so large that progress in that direction is inevitable. It would be great if we had “agents” only in the limited sense that they could use tools, but without the ability to work autonomously on long time-horizon tasks and solve novel problems—like just for a random example, solving the novel problem “how do I make sure humans don’t interfere in my new grand plan?”
Agency (proficient tool use with high reliability for costly actions) might be sufficient to maintain an RSI loop (as an engineer, using settled methodology) even while lacking crucial capabilities (such as coming up with important novel ideas), eventually developing those capabilities without any human input. But even if it works like this, the AI speed advantage might be negated by lack of those capabilities, so that human-led AI research is still faster and mostly bottlenecked by availability and cost of compute.
This is a bit of a separate question, but it’s one I’m very interested in. I think the advantages of general problem-solving abilities will be so large that progress in that direction is inevitable. It would be great if we had “agents” only in the limited sense that they could use tools, but without the ability to work autonomously on long time-horizon tasks and solve novel problems—like just for a random example, solving the novel problem “how do I make sure humans don’t interfere in my new grand plan?”