The worst-case scenario is the case where there are types of intelligence which don’t just look like search, but we don’t know what all of them are; this means that the optimization daemons can be smarter than the search algorithm in a significant sense, creating pressure for our search to find those intelligent agents in order to solve problems more efficiently.
This is exactly what seems most likely to me (see this post).
I think the assumption that all intelligence is search is implausible. For one, current AI systems use methods other than brute force search (e.g. Bayesian inference, gradient descent, MCTS, logic, Q-learning). For another, it seems that there are useful generic cognitive algorithms (e.g. multi-level models, mathematical reasoning, various Bayesian models, various frequentist methods such as confidence intervals) that generalize pretty well across different domains, and are not entirely made of search.
Current AI does stochastic search, but it is still search. Essentially PP complexity class, instead of NP/P. (with a fair amount of domain specific heuristics)
This is exactly what seems most likely to me (see this post).
I think the assumption that all intelligence is search is implausible. For one, current AI systems use methods other than brute force search (e.g. Bayesian inference, gradient descent, MCTS, logic, Q-learning). For another, it seems that there are useful generic cognitive algorithms (e.g. multi-level models, mathematical reasoning, various Bayesian models, various frequentist methods such as confidence intervals) that generalize pretty well across different domains, and are not entirely made of search.
Current AI does stochastic search, but it is still search. Essentially PP complexity class, instead of NP/P. (with a fair amount of domain specific heuristics)
Not all AI is search, and when it is it’s usually not brute force search.