[just now learning about the no free lunch theorem] oh nooo, is this part of the reason so many AI researchers think it’s cool and enlightened to not believe in highly general architectures?
Because they either believe the theorem proves more than it does or because they’re knowingly performing an aestheticised version of it by yowling about how LLMs can’t scale to superintelligence (which is true, but also not a crux).
is this part of the reason so many AI researchers think it’s cool and enlightened to not believe in highly general architectures
I do hear No Free Lunch theorem get thrown around when an architecture fails to solve some problem which its inductive bias doesn’t fit. But I think it’s just thrown around as a vibe.
[just now learning about the no free lunch theorem] oh nooo, is this part of the reason so many AI researchers think it’s cool and enlightened to not believe in highly general architectures?
Because they either believe the theorem proves more than it does or because they’re knowingly performing an aestheticised version of it by yowling about how LLMs can’t scale to superintelligence (which is true, but also not a crux).
I do hear No Free Lunch theorem get thrown around when an architecture fails to solve some problem which its inductive bias doesn’t fit. But I think it’s just thrown around as a vibe.