I’m curious how well you are doing in terms of retaining all the math you have learned. Can you still prove all or most of the theorems in the books you worked through, or do all or most of the exercises in them? How much of it still feels fresh in mind vs something much vaguer that you can only recall in broad strokes? Do you have a reviewing system in place, and if so, what does it look like?
I don’t think about my self-study as “know a bunch of math things”. Rather, it’s a) continually improve at mathematical reasoning, and b) accumulate a bunch of problem-solving strategies and ways-of-looking-at-the-world. I can
consider the analytic properties of a real polynomial (how quickly does the output change with the input? all polynomials are analytic. polynomial approximation theorem)
put on my group theory hat (what kinds of operations does the space of polynomials P admit?)
consider it from a linear algebraic standpoint (P is secretly R∞)
put on a number-theoretic hat (consider the distributional behavior of integer polynomials in a modular context)
think about how polynomials might be used in ML (hypothesis classes, complexity-of-fit, what’s the search space behavior like for SGD if the model space is just the coefficients for degree-50 polynomials?)
think about complexity theory (the nice closure properties of polynomial composition)
or even consider the computability properties of polynomials (this is a bit of a stretch, but… for every decision problem containing a finite amount of YES answers, there exists a polynomial with roots on the base-ten encodings of those YES inputs).
Then, when I actually need to do stuff with polynomials for my research, I can see how the difference of returns from two policies can be represented by a polynomial in the agent’s discount rate, which is a nice result (lemma 32). Insights build on insights.
So, I’m not trying to memorize everything. Leafing through Linear Algebra Done Right, I don’t remember much about what self-adjointness means, or Jordan normal form, or whatever. However, I don’t think that really matters. If I need to use the extraneous stuff, I know it exists, and could just pick it back up.
I am, however, able to regenerate fundamental things I actually use / run into in later studies. I have a mental habit of making myself regenerate random claims in every proof I consider. If we’re relying on the commutativity of addition on the reals, I reflexively supply a proof of that property. I came up with: use the Cauchy sequence limit notion of reals, then rely on the commutativity of rationals under addition for each member of the sequence.
It’s not like I’ve perfectly retained everything I need. I can tell I’m a little rusty on some things. However, I never do conscious reviewing (beyond building on past insights through further study, using the insights in my professional research, and rederiving things periodically). I have empirical feedback that this works pretty well. My PhD qualifier exam involved a matrix analysis question; without having even taken the class, I was able to get the right answer by reasoning using the skills and knowledge I got from self-study.
ETA: FWIW, when I talk with math undergrads at my university about areas we’ve both studied / solve problems with them, my impression is that my comprehension is often better.
I think I would have done well to think more carefully about the benefits of implementing a review system, when you posted this comment. It may have been true that I was getting by with my current study setup, but also accumulating “math recall debt” over time.
I think I underestimated the importance of knowing lots of math facts. For example, I initially replied:
So, I’m not trying to memorize everything. Leafing through Linear Algebra Done Right, I don’t remember much about what self-adjointness means, or Jordan normal form, or whatever. However, I don’t think that really matters. If I need to use the extraneous stuff, I know it exists, and could just pick it back up.
This underestimates the cost of picking it back up, which is linear for each fact in the dependencies I forgot, versus just maintaining the dependencies over time.
For example, when I’m studying quantum mechanics, it seems crazy to say “who cares about self-adjointness”, but not that crazy to say “who cares about self-adjointness” if I’m just doing reinforcement learning theory.
I gestured at another useful habit, but one which only works on skills I use regularly:
I am, however, able to regenerate fundamental things I actually use / run into in later studies. I have a mental habit of making myself regenerate random claims in every proof I consider. If we’re relying on the commutativity of addition on the reals, I reflexively supply a proof of that property. I came up with: use the Cauchy sequence limit notion of reals, then rely on the commutativity of rationals under addition for each member of the sequence.
I couldn’t use this technique to help myself remember what a correlated equilibrium is, because I don’t use concepts which build on ‘correlated equilibria’ very frequently.
I understood correlated equilibria at one point, but I don’t recall anymore, and that makes me sad. Now I’ll go back and put that chain of insights into Anki; if I’d been doing this from the beginning, I wouldn’t have to do that.
I’m curious how well you are doing in terms of retaining all the math you have learned. Can you still prove all or most of the theorems in the books you worked through, or do all or most of the exercises in them? How much of it still feels fresh in mind vs something much vaguer that you can only recall in broad strokes? Do you have a reviewing system in place, and if so, what does it look like?
I don’t think about my self-study as “know a bunch of math things”. Rather, it’s a) continually improve at mathematical reasoning, and b) accumulate a bunch of problem-solving strategies and ways-of-looking-at-the-world. I can
consider the analytic properties of a real polynomial (how quickly does the output change with the input? all polynomials are analytic. polynomial approximation theorem)
put on my group theory hat (what kinds of operations does the space of polynomials P admit?)
consider it from a linear algebraic standpoint (P is secretly R∞)
put on a number-theoretic hat (consider the distributional behavior of integer polynomials in a modular context)
think about how polynomials might be used in ML (hypothesis classes, complexity-of-fit, what’s the search space behavior like for SGD if the model space is just the coefficients for degree-50 polynomials?)
think about complexity theory (the nice closure properties of polynomial composition)
or even consider the computability properties of polynomials (this is a bit of a stretch, but… for every decision problem containing a finite amount of
YESanswers, there exists a polynomial with roots on the base-ten encodings of thoseYESinputs).Then, when I actually need to do stuff with polynomials for my research, I can see how the difference of returns from two policies can be represented by a polynomial in the agent’s discount rate, which is a nice result (lemma 32). Insights build on insights.
So, I’m not trying to memorize everything. Leafing through Linear Algebra Done Right, I don’t remember much about what self-adjointness means, or Jordan normal form, or whatever. However, I don’t think that really matters. If I need to use the extraneous stuff, I know it exists, and could just pick it back up.
I am, however, able to regenerate fundamental things I actually use / run into in later studies. I have a mental habit of making myself regenerate random claims in every proof I consider. If we’re relying on the commutativity of addition on the reals, I reflexively supply a proof of that property. I came up with: use the Cauchy sequence limit notion of reals, then rely on the commutativity of rationals under addition for each member of the sequence.
It’s not like I’ve perfectly retained everything I need. I can tell I’m a little rusty on some things. However, I never do conscious reviewing (beyond building on past insights through further study, using the insights in my professional research, and rederiving things periodically). I have empirical feedback that this works pretty well. My PhD qualifier exam involved a matrix analysis question; without having even taken the class, I was able to get the right answer by reasoning using the skills and knowledge I got from self-study.
ETA: FWIW, when I talk with math undergrads at my university about areas we’ve both studied / solve problems with them, my impression is that my comprehension is often better.
For those reading this thread in the future, Alex has now adopted a more structured approach to reviewing the math he has learned.
I think I would have done well to think more carefully about the benefits of implementing a review system, when you posted this comment. It may have been true that I was getting by with my current study setup, but also accumulating “math recall debt” over time.
I think I underestimated the importance of knowing lots of math facts. For example, I initially replied:
This underestimates the cost of picking it back up, which is linear for each fact in the dependencies I forgot, versus just maintaining the dependencies over time.
For example, when I’m studying quantum mechanics, it seems crazy to say “who cares about self-adjointness”, but not that crazy to say “who cares about self-adjointness” if I’m just doing reinforcement learning theory.
I gestured at another useful habit, but one which only works on skills I use regularly:
I couldn’t use this technique to help myself remember what a correlated equilibrium is, because I don’t use concepts which build on ‘correlated equilibria’ very frequently.
I understood correlated equilibria at one point, but I don’t recall anymore, and that makes me sad. Now I’ll go back and put that chain of insights into Anki; if I’d been doing this from the beginning, I wouldn’t have to do that.