>> Tropical geometry is an interesting, mysterious and reasonable field in mathematics, used for systematically analyzing the asymptotic and “boundary” geometry of polynomial functions and solution sets in high-dimensional spaces, and related combinatorics (it’s actually closely related to my graduate work and some logarithmic algebraic geometry work I did afterwards). It sometimes extends to other interesting asymptotic behaviors (like trees of genetic relatedness). The idea of applying this to partially linear functions appearing in ML is about as silly as trying to see DNA patterns in the arrangement of stars—it’s a total type mismatch.
Shots fired! :D Afaik I’m the only tropical geometry stan in alignment so let me reply to this spicy takedown here.
It’s quite plausible to me that thinking in terms of polytopes, convex is a reasonable and potentially powerful lens on understanding neural networks. Despite the hyperconfident and strong language in this post it seems you agree.
Is it then unreasonable to think that tropical geometry may be relevant too? I don’t think so.
Perhaps your contention is that tropical geometry is more than just thinking in terms of polytopes but specifically the algebraic geometric flavored techniques. Perhaps. I don’t feel strongly about that. If it’s matroids that are most relevant, rather than toric varieties and tropicalized Grassmanians then so be it.
As is mentioned in the text, convex-linear functions are much easier to analyze than general piece-wise linear functions so this decomposition may prove advantageous.
Another direction that may be of interest in this context is the nonsmooth calculus and especially its extension the quasi-differential calculus.
″ as silly trying to see DNA patterns in the arrangement of stars—it’s a total type mismatch”
This statement feels deeply overconfident to me. Whether or not tropical geometry may be relevant to understanding real neural networks can only really be resolved by having a true domain expert ′ commit to the bit’ and research this deeply.
This kind of idle speculation seems not so useful to me.
>> Tropical geometry is an interesting, mysterious and reasonable field in mathematics, used for systematically analyzing the asymptotic and “boundary” geometry of polynomial functions and solution sets in high-dimensional spaces, and related combinatorics (it’s actually closely related to my graduate work and some logarithmic algebraic geometry work I did afterwards). It sometimes extends to other interesting asymptotic behaviors (like trees of genetic relatedness). The idea of applying this to partially linear functions appearing in ML is about as silly as trying to see DNA patterns in the arrangement of stars—it’s a total type mismatch.
Shots fired! :D Afaik I’m the only tropical geometry stan in alignment so let me reply to this spicy takedown here.
It’s quite plausible to me that thinking in terms of polytopes, convex is a reasonable and potentially powerful lens on understanding neural networks. Despite the hyperconfident and strong language in this post it seems you agree.
Is it then unreasonable to think that tropical geometry may be relevant too? I don’t think so.
Perhaps your contention is that tropical geometry is more than just thinking in terms of polytopes but specifically the algebraic geometric flavored techniques. Perhaps. I don’t feel strongly about that. If it’s matroids that are most relevant, rather than toric varieties and tropicalized Grassmanians then so be it.
The basic tropical perspective on deep learning begins by observing ReLU neural networks as ′ tropical rational functions’ , i.e. decomposing the underlying map $f$ of your ReLU neural network as a difference of convex linear functions $f=g-h$. This decomposition isn’t unique, but possibly still quite useful.
As is mentioned in the text, convex-linear functions are much easier to analyze than general piece-wise linear functions so this decomposition may prove advantageous.
Another direction that may be of interest in this context is the nonsmooth calculus and especially its extension the quasi-differential calculus.
″ as silly trying to see DNA patterns in the arrangement of stars—it’s a total type mismatch”
This statement feels deeply overconfident to me. Whether or not tropical geometry may be relevant to understanding real neural networks can only really be resolved by having a true domain expert ′ commit to the bit’ and research this deeply.
This kind of idle speculation seems not so useful to me.