Yeah, I agree that some inductive bias is probably necessary. But not all inductive biases are equal; some are much more general than others, and in particular I claim that ‘narrow’ inductive biases (e.g. specializing architectures to match domains) probably have net ~zero benefit compared to those learned from data
Yeah, I agree that some inductive bias is probably necessary. But not all inductive biases are equal; some are much more general than others, and in particular I claim that ‘narrow’ inductive biases (e.g. specializing architectures to match domains) probably have net ~zero benefit compared to those learned from data