You got to the end of the essay and went “down” into the details instead of “up” in to the larger problem. Going up would be productive I think, because this is an issue that sort of comes up with every single field of human knowledge that exists, especially the long tail of specializations and options for graduate studies.
When you were an undergraduate and spent a lot of time thinking about the structure of mathematical knowledge, you were building a sort of map of all the maps that exist inside the books and minds of of the community of mathematians, with cues based on presumed structure in math itself, that everyone was studying in common.
Your “metamap of math” that you had in your head is not something shared by everyone, and I do not think that it would be easy for you to share (though maybe I’m wrong about your assessment of its sharability).
When I saw title of your post, I thought to myself “Yes! I want a map of all of human knowledge too!” and I was hoping that I’d get pointers towards a generalization of your undergrad work in mathematics, except written down, and shareable, and about “all of human knowledge”. But then I got to the end of your essay and, as I said, it went “down” instead of “up”… :-/
Anyway, for deep learning, I think a lot of the metamap comes from just running code to get a practical feel for it, because unlike math the field is more empirically based (where people try lots of stuff, hide their mistakes, and then polish up the best thing they found to present as if they understand exactly how and why it worked).
For myself, I watch for papers to float by, and look for code to download and try out, and personally I get a lot from googelstalking smart people via blogs.
The best thing I know in this vein as a starting point is a post by Ilya Sutskever (guest writing on Yisong Yue’s blog), with an overview of practical issues in deep learning to keep in mind when trying to make models train and do well that seem not to click when you think you have enough data and enough GPU and a decent architecture, yet they are still not working.
You got to the end of the essay and went “down” into the details instead of “up” in to the larger problem. Going up would be productive I think, because this is an issue that sort of comes up with every single field of human knowledge that exists, especially the long tail of specializations and options for graduate studies.
When you were an undergraduate and spent a lot of time thinking about the structure of mathematical knowledge, you were building a sort of map of all the maps that exist inside the books and minds of of the community of mathematians, with cues based on presumed structure in math itself, that everyone was studying in common.
Your “metamap of math” that you had in your head is not something shared by everyone, and I do not think that it would be easy for you to share (though maybe I’m wrong about your assessment of its sharability).
When I saw title of your post, I thought to myself “Yes! I want a map of all of human knowledge too!” and I was hoping that I’d get pointers towards a generalization of your undergrad work in mathematics, except written down, and shareable, and about “all of human knowledge”. But then I got to the end of your essay and, as I said, it went “down” instead of “up”… :-/
Anyway, for deep learning, I think a lot of the metamap comes from just running code to get a practical feel for it, because unlike math the field is more empirically based (where people try lots of stuff, hide their mistakes, and then polish up the best thing they found to present as if they understand exactly how and why it worked).
For myself, I watch for papers to float by, and look for code to download and try out, and personally I get a lot from googelstalking smart people via blogs.
The best thing I know in this vein as a starting point is a post by Ilya Sutskever (guest writing on Yisong Yue’s blog), with an overview of practical issues in deep learning to keep in mind when trying to make models train and do well that seem not to click when you think you have enough data and enough GPU and a decent architecture, yet they are still not working.