Hello from the future! I’m interested to hear how your views have updated since this comment and post were written. 1. What is your credence that the brain learns via gradient descent? 2. What is your credence that it in fact does so in a way relevantly similar to backprop? 3. Do you still think that insofar as your credence in 1 is high, timelines are short?
The sad and honest truth, though, is that since I wrote this post, I haven’t thought about it. :( I haven’t picked up on any key new piece of evidence—though I also haven’t been looking.
I could give you credences, but that would mostly just involve rereading this and loading up all the thoughts
Ok! Well, FWIW, it seems very likely to me that the brain learns via gradient descent, and indeed probable that it does something relevantly similar (though of course not identical to) backprop. (See the link above). But I feel very much an imposter discussing all this stuff since I lack technical expertise. I’d be interested to hear your take on this stuff sometime if you have one or want to make one! See also:
Hello from the future! I’m interested to hear how your views have updated since this comment and post were written. 1. What is your credence that the brain learns via gradient descent? 2. What is your credence that it in fact does so in a way relevantly similar to backprop? 3. Do you still think that insofar as your credence in 1 is high, timelines are short?
I appreciate you following up on this!
The sad and honest truth, though, is that since I wrote this post, I haven’t thought about it. :( I haven’t picked up on any key new piece of evidence—though I also haven’t been looking.
I could give you credences, but that would mostly just involve rereading this and loading up all the thoughts
Ok! Well, FWIW, it seems very likely to me that the brain learns via gradient descent, and indeed probable that it does something relevantly similar (though of course not identical to) backprop. (See the link above). But I feel very much an imposter discussing all this stuff since I lack technical expertise. I’d be interested to hear your take on this stuff sometime if you have one or want to make one! See also:
https://arxiv.org/abs/2006.04182 (Brains = predictive processing = backprop = artificial neural nets)
https://www.biorxiv.org/content/10.1101/764258v2.full (IIRC this provides support for Kaplan’s view that human ability to extrapolate is really just interpolation done by a bigger brain on more and better data.)
I’m currently on vacation, but I’d be interested in setting up a call once I’m back in 2 weeks! :) I’ll send you my calendly in PM