Thinking of the cortical columns as models in an ensemble… Have ML people tried ensemble models with tens of thousands of models? If so, are they substantially better than using only a few dozen? If they aren’t, then why does the brain need so many?
According to A Recipe for Training NNs, model ensembles stop being helpful at ~5 models. But that’s when they all have the same inputs and outputs. The more brain-like thing is to have lots of models whose inputs comprise various different subsets of both the inputs and the other models’ outputs.
...But then, you don’t really call it an “ensemble”, you call it a “bigger more complicated neural architecture”, right? I mean, I can take a deep NN and call it “six different models, where the output of model #1 is the input of model #2 etc.”, but no one in ML would say that, they would call it a single six-layer model...
Interesting, thanks!
Thinking of the cortical columns as models in an ensemble… Have ML people tried ensemble models with tens of thousands of models? If so, are they substantially better than using only a few dozen? If they aren’t, then why does the brain need so many?
According to A Recipe for Training NNs, model ensembles stop being helpful at ~5 models. But that’s when they all have the same inputs and outputs. The more brain-like thing is to have lots of models whose inputs comprise various different subsets of both the inputs and the other models’ outputs.
...But then, you don’t really call it an “ensemble”, you call it a “bigger more complicated neural architecture”, right? I mean, I can take a deep NN and call it “six different models, where the output of model #1 is the input of model #2 etc.”, but no one in ML would say that, they would call it a single six-layer model...