Thank you for writing this! I once thought about asking LW for something like this but never got around to it.
I’m an undergraduate; I expect to take several more late-undergraduate- to early-graduate-level math courses. Presumably some will turn out to be much more valuable to me than others, and presumably this is possible to predict better-than-randomly in advance. Do you [or anyone else] have thoughts on how to choose between math courses other than those you mention, either specific courses (and why they might be valuable) or general principles (any why they seem reasonable)? (I don’t have any sense of what the math of agency and alignment is like, and I hope to get a feel for it sometime in the next year, but I can’t right now — by the way, any recommendations on how to do that?)
I expect to take several more late-undergraduate- to early-graduate-level math courses. Presumably some will turn out to be much more valuable to me than others, and presumably this is possible to predict better-than-randomly in advance.
Yes, but not in a uniform way. The mathematical frontier is so large, and semesters so short, that Professor A’s version of, for instance, a grad level “Dynamical Systems” course can have literally no overlap with Professor B’s version. Useful advice here is going to have to come from Professors A and B (though not necessarily directly).
what the math of agency … is like
Underdeveloped. There’s some interesting work coming out of the programming language theory / applied category theory region these days (Neil Ghani and David Spivak come to mind), but “the math of agency” is not even an identifiable field yet, let alone one mature enough to show up in curricula.
I don’t have recommendations for courses or principles to select them beyond what’s in the post. (Otherwise I would have put them in the post.)
I don’t have any sense of what the math of agency and alignment is like, and I hope to get a feel for it sometime in the next year, but I can’t right now — by the way, any recommendations on how to do that?
I don’t think you’re going to find anybody with existing good answers. The embedded agency sequence is the best articulation of the problems which I currently know of. (Even there I disagree with the degree of emphasis placed on various subproblems/frames, but it is nonetheless very good.)
If you want a useful tarting point to think about these things yourself: ask how to calculate the world-model and preferences of an e-coli directly from a low-level specification of the cell (i.e. all the reaction dynamics and concentrations and forces and whatnot).
Thank you for writing this! I once thought about asking LW for something like this but never got around to it.
I’m an undergraduate; I expect to take several more late-undergraduate- to early-graduate-level math courses. Presumably some will turn out to be much more valuable to me than others, and presumably this is possible to predict better-than-randomly in advance. Do you [or anyone else] have thoughts on how to choose between math courses other than those you mention, either specific courses (and why they might be valuable) or general principles (any why they seem reasonable)? (I don’t have any sense of what the math of agency and alignment is like, and I hope to get a feel for it sometime in the next year, but I can’t right now — by the way, any recommendations on how to do that?)
Yes, but not in a uniform way. The mathematical frontier is so large, and semesters so short, that Professor A’s version of, for instance, a grad level “Dynamical Systems” course can have literally no overlap with Professor B’s version. Useful advice here is going to have to come from Professors A and B (though not necessarily directly).
Underdeveloped. There’s some interesting work coming out of the programming language theory / applied category theory region these days (Neil Ghani and David Spivak come to mind), but “the math of agency” is not even an identifiable field yet, let alone one mature enough to show up in curricula.
I don’t have recommendations for courses or principles to select them beyond what’s in the post. (Otherwise I would have put them in the post.)
I don’t think you’re going to find anybody with existing good answers. The embedded agency sequence is the best articulation of the problems which I currently know of. (Even there I disagree with the degree of emphasis placed on various subproblems/frames, but it is nonetheless very good.)
If you want a useful tarting point to think about these things yourself: ask how to calculate the world-model and preferences of an e-coli directly from a low-level specification of the cell (i.e. all the reaction dynamics and concentrations and forces and whatnot).