The orthogonality thesis, which assumes that goals and core intelligence of an AGI system can be cleanly factored apart, has been challenged by recent developments in machine learning (ML). Model-free reinforcement learning (RL) systems, unlike their well-factored counterparts, learn policies or value functions directly from the reward signal, leading to less general agents. These non-factored agents are favored in contemporary ML because they are more efficient, amortizing the cost of planning and exploiting problem domain structure.
However, the tradeoff between specificity and generality, a consequence of the no-free-lunch theorem, means that achieving full generality is prohibitively expensive. Instead of viewing full orthogonality as the default or ideal case, it should be considered one end of a pareto tradeoff curve, with different architectures occupying various positions along it.
The future of AGI systems will be shaped by the slope of the pareto frontier across the range of general capabilities, determining whether we see fully general AGI singletons, multiple general systems, or a large number of highly specialized systems.
Let me know which you prefer.
[I adapted some parts of −4′s summary that it seemed weren’t sufficiently covered in my original account.]
FWIW, here’s a summary courtesy of −4:
Let me know which you prefer.
[I adapted some parts of −4′s summary that it seemed weren’t sufficiently covered in my original account.]