(Note that I’m not making a claim about how search is central to human capabilities relative to other species; I’m just saying search is useful in general. Plausibly also for other species, though it is more obvious for humans)
From my POV, the “cultural intelligence hypothesis” is not a counterpoint to importance of search. It’s obvious that culture is important for human capabilities, but it also seems obvious to me that search is important. Building printing presses or steam engines is not something that a bundle of heuristics can do, IMO, without gaining those heuristics via a long process of evolutionary trial-and-error. And it seems important that humans can build steam engines without generations of breeding better steam-engine-engineers.
Re AlphaStar and AlphaZero: I’ve never played Starcraft, so I don’t have good intuitions for what capabilities are needed. But on the definitions of search that I use, the AlphaZero policy network definitely performs search. In fact out of current systems it’s probably the one that most clearly performs search!
...Now I’m wondering whether our disagreement just comes from having different definitions of search in mind. Skimming your other comments above, it seems like you take a more narrow view of search = literally iterating through solutions and picking a good one. This is fine by me definitionally, but I don’t think the fact that models will not learn search(narrow) is very interesting for alignment, or has the implications that you list in the post? Though ofc I might still be misunderstanding you here.
Yeah it’s probably definitions. With the caveat that I don’t mean the narrow “literallyiteratesover solutions”, but roughly “behaves (especially off the training distribution) as if it’s iterating over solutions”, like Abram Demski’s term selection.
(Note that I’m not making a claim about how search is central to human capabilities relative to other species; I’m just saying search is useful in general. Plausibly also for other species, though it is more obvious for humans)
From my POV, the “cultural intelligence hypothesis” is not a counterpoint to importance of search. It’s obvious that culture is important for human capabilities, but it also seems obvious to me that search is important. Building printing presses or steam engines is not something that a bundle of heuristics can do, IMO, without gaining those heuristics via a long process of evolutionary trial-and-error. And it seems important that humans can build steam engines without generations of breeding better steam-engine-engineers.
Re AlphaStar and AlphaZero: I’ve never played Starcraft, so I don’t have good intuitions for what capabilities are needed. But on the definitions of search that I use, the AlphaZero policy network definitely performs search. In fact out of current systems it’s probably the one that most clearly performs search!
...Now I’m wondering whether our disagreement just comes from having different definitions of search in mind. Skimming your other comments above, it seems like you take a more narrow view of search = literally iterating through solutions and picking a good one. This is fine by me definitionally, but I don’t think the fact that models will not learn search(narrow) is very interesting for alignment, or has the implications that you list in the post? Though ofc I might still be misunderstanding you here.
Yeah it’s probably definitions. With the caveat that I don’t mean the narrow “literally iterates over solutions”, but roughly “behaves (especially off the training distribution) as if it’s iterating over solutions”, like Abram Demski’s term selection.