If a field involves math, and you cannot do the math, you are not qualified to comment on that field.
+1 to this post.
The basic math of neural networks
Learn about first and second derivatives and finding a maximum of a function. Then think about how you might find a maximum if you can only make little hops at a time.
Learn a little linear algebra (what a matrix inverse, determinant, etc. is). Understand the relationship between solving a system of linear equations and matrix inverse. Then think about what you might want to do if you have more equations than unknowns (can’t invert exactly but can find something that’s “as close to an inverse as possible” in some sense). A huge chunk of stuff that falls under the heading of “statistics/machine learning/neural networks/etc” is basically variations of that idea.
Some common computer science algorithms
Read Structure and Interpretation of Computer Programs: one of the highest concept/page density for computer science books.
Important algorithmic ideas are, in my opinion: hashing, dynamic programming/memoization, divide and conquer by recursion, splitting up tasks to be done in parallel, and locality (things you want at a particular point are often close in space and time).
Locality is sort of like “a smoothness assumption on access.” The reason your laptop is fast even though your hard disk is slow is due to locality being generally true.
“I will always link to my ingroup”, says Scott. So it is with me: I always recommend learning about association vs causation. If you are into learning by doing, try to find some media articles that make claims of the form “scientists report that to [Y], do [X],” and look up the original study and think about if the media claim actually follows (it generally does not). This will also give you practice reading empirical papers, which is a good skill to have. Stuff the authors do in such papers isn’t magic, after all: the set of statistical ideas that come up over and over again in them is fairly small.
he was also a literal child prodigy
Don’t think like that. There are no wizards, just people doing sensible things.
+1 for Structure and Interpretation of Computer Programs (aka SICP, aka “the wizard book”) - this is a legendary programming book. Here is an interactive version: https://xuanji.appspot.com/isicp/.
I also agree on the important algorithmic ideas, with one addition: algorithmic analysis. Just as you can describe the movement of the planets with a few simple equations, and that’s beautiful, you can describe any sequence of steps to finish a task as an algorithm. And you can mathematically analyze the efficiency of that sequence: as the task gets larger, do the number of steps required to finish it grow linearly, quadratically, logarithmically (we hope)? This is a broadly applicable and powerful idea, since pretty much everything (even learning) involves a sequence of steps or process.
Just as you can describe the movement of the planets with a few simple equations
and
algorithmic analysis
Will you make top level comments for both of these, so that people can vote on them? (I can do it, but I figure you should get any karma from the upvotes)
How flattering; I’ve now done so. Also, I very much like your approach to learning math by grounding it in concrete subjects. Many people say they learned calculus best by learning it alongside physics, since calculus appears much more concrete when you look at the velocity and arc of, say, a fired cannonball.
Finally, here’s an excellent article from Barbara Oakley, who learned math starting about age 26 after getting out of the Army. She’s now an engineering prof, and teaches a MOOC called “Learning How to Learn” (I have not taken it, but I have reviewed the topics, and it appears to hit all the correct points): http://nautil.us/issue/17/big-bangs/how-i-rewired-my-brain-to-become-fluent-in-math-rd
+1 to this post.
Learn about first and second derivatives and finding a maximum of a function. Then think about how you might find a maximum if you can only make little hops at a time.
Learn a little linear algebra (what a matrix inverse, determinant, etc. is). Understand the relationship between solving a system of linear equations and matrix inverse. Then think about what you might want to do if you have more equations than unknowns (can’t invert exactly but can find something that’s “as close to an inverse as possible” in some sense). A huge chunk of stuff that falls under the heading of “statistics/machine learning/neural networks/etc” is basically variations of that idea.
Read Structure and Interpretation of Computer Programs: one of the highest concept/page density for computer science books.
Important algorithmic ideas are, in my opinion: hashing, dynamic programming/memoization, divide and conquer by recursion, splitting up tasks to be done in parallel, and locality (things you want at a particular point are often close in space and time).
Locality is sort of like “a smoothness assumption on access.” The reason your laptop is fast even though your hard disk is slow is due to locality being generally true.
“I will always link to my ingroup”, says Scott. So it is with me: I always recommend learning about association vs causation. If you are into learning by doing, try to find some media articles that make claims of the form “scientists report that to [Y], do [X],” and look up the original study and think about if the media claim actually follows (it generally does not). This will also give you practice reading empirical papers, which is a good skill to have. Stuff the authors do in such papers isn’t magic, after all: the set of statistical ideas that come up over and over again in them is fairly small.
Don’t think like that. There are no wizards, just people doing sensible things.
+1 for Structure and Interpretation of Computer Programs (aka SICP, aka “the wizard book”) - this is a legendary programming book. Here is an interactive version: https://xuanji.appspot.com/isicp/.
I also agree on the important algorithmic ideas, with one addition: algorithmic analysis. Just as you can describe the movement of the planets with a few simple equations, and that’s beautiful, you can describe any sequence of steps to finish a task as an algorithm. And you can mathematically analyze the efficiency of that sequence: as the task gets larger, do the number of steps required to finish it grow linearly, quadratically, logarithmically (we hope)? This is a broadly applicable and powerful idea, since pretty much everything (even learning) involves a sequence of steps or process.
and
Will you make top level comments for both of these, so that people can vote on them? (I can do it, but I figure you should get any karma from the upvotes)
How flattering; I’ve now done so. Also, I very much like your approach to learning math by grounding it in concrete subjects. Many people say they learned calculus best by learning it alongside physics, since calculus appears much more concrete when you look at the velocity and arc of, say, a fired cannonball.
Finally, here’s an excellent article from Barbara Oakley, who learned math starting about age 26 after getting out of the Army. She’s now an engineering prof, and teaches a MOOC called “Learning How to Learn” (I have not taken it, but I have reviewed the topics, and it appears to hit all the correct points): http://nautil.us/issue/17/big-bangs/how-i-rewired-my-brain-to-become-fluent-in-math-rd
Thank you!