I’d say (b) -- it seems quite unlikely to me that the NTK/GP are universally data-efficient, while neural nets might be(although that’s mostly speculation on my part). I think the lack of feature learning is a stronger argument that NTK/GP don’t characterize neural nets well.
Yeah, that summary sounds right.
I’d say (b) -- it seems quite unlikely to me that the NTK/GP are universally data-efficient, while neural nets might be(although that’s mostly speculation on my part). I think the lack of feature learning is a stronger argument that NTK/GP don’t characterize neural nets well.