Sorry for the late reply! Do you mind sharing a ref for Critch’s new work? I have tried to find something about boundaries but was unsuccessful.
As for the historical accident, I would situate it more around the 17th century, when the theory of mechanics was roughly as advanced as that of agency. I don’t feel that goals and values require much more advanced math, only math as new as differential calculus was at the time.
Though we now have many pieces that seem to aim in the right direction (variational calculus in general, John Baez and colleagues’ investigations of blackboxing via category theory...), it seems more by chance than by concerted, literature-wide effort. But I do hope to build on these pieces.
Category theory was developed in the 50s from considerations in algebraic topology. Algebraic topology was an extremely technically sophisticated field already in the 50s (and has by now reached literally incredible abstract heights).
I suppose one could imagine an alternate world where Galois invents category theory but it seems apparent to me that the amount of long-term development significantly divorced from direct applications (as calculus was) was needed for category theory to spring up and mature—indeed it is still in its teenage rebellious phase in the grand scheme of things!
John Baez et al ’s work on Blackboxing is uses several ideas only developed recently and though I am a fan of abstract mathematics in general and category theory in particular I hasten to say this work is only a small part of a big and as of yet mostly unfinished story.
I’m optimistic that this might eventually lead to serious insights in understanding agency but at the moment it seems we are still quite far.
A more directed search for agent foundations math is needed, happening (but we need much more!) and likely to bear fruits on the medium-short term but I suspect many of the ingredients are likely things that have been developed with very different motivations in mind.
Thanks for your thoughts and for the link! I definitely agree that we are very far from practical category-inspired improvements at this stage; I simply wonder whether there isn’t something fundamentally as simple and novel as differential equations waiting around the corner and that we are taking a very circuitous route toward through very deep metamathematics! (Baez’s rosetta stone paper and work by Abramsky and Coeck on quantum logic have convinced me that we need something like “not being in a Cartesian category” to account for notions like context and meaning, but that quantum stuff is only one step removed from the most Cartesian classical logic/physics and we probably need to go to the other extreme to find a different kind of simplicity)
Sorry for the late reply! Do you mind sharing a ref for Critch’s new work? I have tried to find something about boundaries but was unsuccessful.
As for the historical accident, I would situate it more around the 17th century, when the theory of mechanics was roughly as advanced as that of agency. I don’t feel that goals and values require much more advanced math, only math as new as differential calculus was at the time.
Though we now have many pieces that seem to aim in the right direction (variational calculus in general, John Baez and colleagues’ investigations of blackboxing via category theory...), it seems more by chance than by concerted, literature-wide effort. But I do hope to build on these pieces.
Category theory was developed in the 50s from considerations in algebraic topology. Algebraic topology was an extremely technically sophisticated field already in the 50s (and has by now reached literally incredible abstract heights).
I suppose one could imagine an alternate world where Galois invents category theory but it seems apparent to me that the amount of long-term development significantly divorced from direct applications (as calculus was) was needed for category theory to spring up and mature—indeed it is still in its teenage rebellious phase in the grand scheme of things!
John Baez et al ’s work on Blackboxing is uses several ideas only developed recently and though I am a fan of abstract mathematics in general and category theory in particular I hasten to say this work is only a small part of a big and as of yet mostly unfinished story. I’m optimistic that this might eventually lead to serious insights in understanding agency but at the moment it seems we are still quite far.
A more directed search for agent foundations math is needed, happening (but we need much more!) and likely to bear fruits on the medium-short term but I suspect many of the ingredients are likely things that have been developed with very different motivations in mind.
Edit: see https://www.lesswrong.com/s/LWJsgNYE8wzv49yEc for Critch’s new work on Boundaries
Thanks for your thoughts and for the link! I definitely agree that we are very far from practical category-inspired improvements at this stage; I simply wonder whether there isn’t something fundamentally as simple and novel as differential equations waiting around the corner and that we are taking a very circuitous route toward through very deep metamathematics! (Baez’s rosetta stone paper and work by Abramsky and Coeck on quantum logic have convinced me that we need something like “not being in a Cartesian category” to account for notions like context and meaning, but that quantum stuff is only one step removed from the most Cartesian classical logic/physics and we probably need to go to the other extreme to find a different kind of simplicity)
No problem!
Do you mean monoidal categories? I think that’s what the central concept in the Abramsly-Coeke work & the Baez Rosetta stone paper is.