Your view may have a surprising implication: Instead of pushing for an AI pause, perhaps we should work hard to encourage the commercialization of current approaches.
If you believe that LLMs aren’t a path to full AGI, successful LLM commercialization means that LLMs eat low-hanging fruit and crowd out competing approaches which could be more dangerous. It’s like spreading QWERTY as a standard if you want everyone to type a little slower. If tons of money and talent is pouring into an AI approach that’s relatively neutered and easy to align, that could actually be a good thing.
A toy model: Imagine an economy where there are 26 core tasks labeled from A to Z, ordered from easy to hard. You’re claiming that LLMs + CoT provide a path to automate tasks A through Q, but fundamental limitations mean they’ll never be able to automate tasks R through Z. To automate jobs R through Z would require new, dangerous core dynamics. If we succeed in automating A through Q with LLMs, that reduces the economic incentive to develop more powerful techniques that work for the whole alphabet. It makes it harder for new techniques to gain a foothold, since the easy tasks already have incumbent players. Additionally, it will take some time for LLMs to automate tasks A through Q, and that buys time for fundamental alignment work.
From a policy perspective, an obvious implication is to heavily tax basic AI research, but have a more favorable tax treatment for applications work (and interpretability work?) That encourages AI companies to allocate workers away from dangerous new ideas and towards applications work. People argue that policymakers can’t tell apart good alignment schemes and bad alignment schemes. Differentiating basic research from applications work seems a lot easier.
A lot of people in the community want to target big compute clusters run by big AI companies, but I’m concerned that will push researchers to find alternative, open-source approaches with dangerous/unstudied core dynamics. “If it ain’t broke, don’t fix it.” If you think current popular approaches are both neutered and alignable, you should be wary of anything which disrupts the status quo.
(Of course, this argument could fail if successful commercialization just increases the level of “AI hype”, where “AI hype” also inevitably translates into more basic research, e.g. as people migrate from other STEM fields towards AI. I still think it’s an argument worth considering though.)
That’s not surprising to me! I pretty much agree with all of this, yup. I’d only add that:
This is why I’m fairly unexcited about the current object-level regulation, and especially the “responsible scaling policies”. Scale isn’t what matters, novel architectural advances is. Scale is safe, and should be encouraged; new theoretical research is dangerous and should be banned/discouraged.
The current major AI labs are fairly ideological about getting to AGI specifically. If they actually pivoted to just scaling LLMs, that’d be great, but I don’t think they’d do it by default.
I agree that LLMs aren’t dangerous. But that’s entirely separate from whether they’re a path to real AGI that is. I think adding self-directed learning and agency to LLMs by using them in cognitive architectures is relatively straightforward: Capabilities and alignment of LLM cognitive architectures.
On this model, improvements in LLMs do contribute to dangerous AGI. They need the architectural additions as well, but better LLMs make those easier.
Your view may have a surprising implication: Instead of pushing for an AI pause, perhaps we should work hard to encourage the commercialization of current approaches.
If you believe that LLMs aren’t a path to full AGI, successful LLM commercialization means that LLMs eat low-hanging fruit and crowd out competing approaches which could be more dangerous. It’s like spreading QWERTY as a standard if you want everyone to type a little slower. If tons of money and talent is pouring into an AI approach that’s relatively neutered and easy to align, that could actually be a good thing.
A toy model: Imagine an economy where there are 26 core tasks labeled from A to Z, ordered from easy to hard. You’re claiming that LLMs + CoT provide a path to automate tasks A through Q, but fundamental limitations mean they’ll never be able to automate tasks R through Z. To automate jobs R through Z would require new, dangerous core dynamics. If we succeed in automating A through Q with LLMs, that reduces the economic incentive to develop more powerful techniques that work for the whole alphabet. It makes it harder for new techniques to gain a foothold, since the easy tasks already have incumbent players. Additionally, it will take some time for LLMs to automate tasks A through Q, and that buys time for fundamental alignment work.
From a policy perspective, an obvious implication is to heavily tax basic AI research, but have a more favorable tax treatment for applications work (and interpretability work?) That encourages AI companies to allocate workers away from dangerous new ideas and towards applications work. People argue that policymakers can’t tell apart good alignment schemes and bad alignment schemes. Differentiating basic research from applications work seems a lot easier.
A lot of people in the community want to target big compute clusters run by big AI companies, but I’m concerned that will push researchers to find alternative, open-source approaches with dangerous/unstudied core dynamics. “If it ain’t broke, don’t fix it.” If you think current popular approaches are both neutered and alignable, you should be wary of anything which disrupts the status quo.
(Of course, this argument could fail if successful commercialization just increases the level of “AI hype”, where “AI hype” also inevitably translates into more basic research, e.g. as people migrate from other STEM fields towards AI. I still think it’s an argument worth considering though.)
That’s not surprising to me! I pretty much agree with all of this, yup. I’d only add that:
This is why I’m fairly unexcited about the current object-level regulation, and especially the “responsible scaling policies”. Scale isn’t what matters, novel architectural advances is. Scale is safe, and should be encouraged; new theoretical research is dangerous and should be banned/discouraged.
The current major AI labs are fairly ideological about getting to AGI specifically. If they actually pivoted to just scaling LLMs, that’d be great, but I don’t think they’d do it by default.
I agree that LLMs aren’t dangerous. But that’s entirely separate from whether they’re a path to real AGI that is. I think adding self-directed learning and agency to LLMs by using them in cognitive architectures is relatively straightforward: Capabilities and alignment of LLM cognitive architectures.
On this model, improvements in LLMs do contribute to dangerous AGI. They need the architectural additions as well, but better LLMs make those easier.