The statement seems almost tautological – couldn’t we somewhat similarly claim that we’ll understand NNs in roughly the same ways that we understand houses, except where we have reasons to think otherwise? The “except where we have reasons to think otherwise” bit seems to be doing a lot of work.
I agree that the phrasing could be better; any suggestions?
I actually think you could just drop that intro altogether, or move it later into the post. We do have pretty good evidence of modularity in the brain (as well as other biological systems) and in trained neural nets; it seems to be a pretty common property of large systems “evolved” by local optimization. And the rest of the post (as well as some of the other comments) does a good job of talking about some of that evidence. It’s a good post, and I think the arguments later in the post are stronger than that opening.
(On the other hand, if you’re opening with it because that was your own main prior, then that makes sense. In that case, maybe note that it was a prior for you, but that the evidence from other directions is strong enough that we don’t need to rely much on that prior?)
Thanks, that’s helpful. I do think there’s a weak version of this which is an important background assumption for the post (e.g. without that assumption I’d need to explain the specific ways in which ANNs and BNNs are similar), so I’ve now edited the opening lines to convey that weak version instead. (I still believe the original version but agree that it’s not worth defending here.)
Yeah, I’m not trying to say that the point is invalid, just that phrasing may give the point more appeal than is warranted from being somewhat in the direction of a deepity. Hmm, I’m not sure what better phrasing would be.
The statement seems almost tautological – couldn’t we somewhat similarly claim that we’ll understand NNs in roughly the same ways that we understand houses, except where we have reasons to think otherwise? The “except where we have reasons to think otherwise” bit seems to be doing a lot of work.
Compare: when trying to predict events, you should use their base rate except when you have specific updates to it.
Similarly, I claim, our beliefs about brains should be the main reference for our beliefs about neural networks, which we can then update from.
I agree that the phrasing could be better; any suggestions?
I actually think you could just drop that intro altogether, or move it later into the post. We do have pretty good evidence of modularity in the brain (as well as other biological systems) and in trained neural nets; it seems to be a pretty common property of large systems “evolved” by local optimization. And the rest of the post (as well as some of the other comments) does a good job of talking about some of that evidence. It’s a good post, and I think the arguments later in the post are stronger than that opening.
(On the other hand, if you’re opening with it because that was your own main prior, then that makes sense. In that case, maybe note that it was a prior for you, but that the evidence from other directions is strong enough that we don’t need to rely much on that prior?)
Thanks, that’s helpful. I do think there’s a weak version of this which is an important background assumption for the post (e.g. without that assumption I’d need to explain the specific ways in which ANNs and BNNs are similar), so I’ve now edited the opening lines to convey that weak version instead. (I still believe the original version but agree that it’s not worth defending here.)
Yeah, I’m not trying to say that the point is invalid, just that phrasing may give the point more appeal than is warranted from being somewhat in the direction of a deepity. Hmm, I’m not sure what better phrasing would be.