The claim that specialized machines always beat general ones seems questionable in the context of an AGI. Actually, I’m not sure I understand the claim in the first place. Maybe he means by analogy to a supervised learning system—if you take a network trained to recognize cat pictures, and also train it to recognize dog pictures, then given a fixed number of parameters you can expect it will get less good at recognizing cat pictures.
The claim that specialized machines always beat general ones seems questionable in the context of an AGI. Actually, I’m not sure I understand the claim in the first place. Maybe he means by analogy to a supervised learning system—if you take a network trained to recognize cat pictures, and also train it to recognize dog pictures, then given a fixed number of parameters you can expect it will get less good at recognizing cat pictures.