I am clearly in the skeptic camp, in the sense that I don’t believe the current architecture will get to AGI with our resources. That is if all the GPU, training data in the world where used it wouldn’t be sufficient and maybe no amount of compute/data would.
To me the strongest evidence that our architecture doesn’t learn and generalize well isn’t LLM but in fact Tesla autopilot. It has ~10,000* more training data than a person, much more FLOPS and is still not human level. I think Tesla is doing pretty much everything major right with their training setup. Our current AI setups just don’t learn or generalize as well as the human brain and similar. They don’t extract symbols or diverse generalizations from high bandwidth un-curated data like video. Scaffolding doesn’t change this.
A medium term but IMO pretty much guaranteed way to get this would be to study and fully characterize the cortical column in the human/mammal brain.
I am clearly in the skeptic camp, in the sense that I don’t believe the current architecture will get to AGI with our resources. That is if all the GPU, training data in the world where used it wouldn’t be sufficient and maybe no amount of compute/data would.
To me the strongest evidence that our architecture doesn’t learn and generalize well isn’t LLM but in fact Tesla autopilot. It has ~10,000* more training data than a person, much more FLOPS and is still not human level. I think Tesla is doing pretty much everything major right with their training setup. Our current AI setups just don’t learn or generalize as well as the human brain and similar. They don’t extract symbols or diverse generalizations from high bandwidth un-curated data like video. Scaffolding doesn’t change this.
A medium term but IMO pretty much guaranteed way to get this would be to study and fully characterize the cortical column in the human/mammal brain.