Has anyone involved put any effort into falsifying this hypothesis in concrete terms and is offering some kind of bold bet?
Well, the “Act 1” project has the following under “What are the most likely causes and outcomes if this project fails?”:
Other risks include a failure to generalize:
Emergent behaviors are already noticed by people developing multi-agent systems and trained or otherwise optimized out, and the behaviors found at the GPT-4 level of intelligence do not scale to the next-generation of models
Failure to incorporate agents being developed by independent third-party developers and understand how they work, and diverge significantly from raw models being used
The previously mentioned notion that the “simulators” framing will remain the correct-in-the-limit description of what ML models are could also be viewed as a bold prediction they’re making.
From my point of view, the latter is really the main issue here. I think all the near-anthropomorphization is basically fine and accurate as long as they’re studying the metaphorical “smiley face” on the “shoggoth”, and how that face’s features and expressions change in response to prompts. But in the eventuality that we move outside the “mask-and-shoggoth” paradigm, all of these principles would fall away, and I’ve never seen any strong arguments that we won’t (the ever-popular “straight lines on graphs” is unconvincing).
Well, the “Act 1” project has the following under “What are the most likely causes and outcomes if this project fails?”:
The previously mentioned notion that the “simulators” framing will remain the correct-in-the-limit description of what ML models are could also be viewed as a bold prediction they’re making.
From my point of view, the latter is really the main issue here. I think all the near-anthropomorphization is basically fine and accurate as long as they’re studying the metaphorical “smiley face” on the “shoggoth”, and how that face’s features and expressions change in response to prompts. But in the eventuality that we move outside the “mask-and-shoggoth” paradigm, all of these principles would fall away, and I’ve never seen any strong arguments that we won’t (the ever-popular “straight lines on graphs” is unconvincing).