It’s interesting that you mention that; I had never even considered applying the combination blindly to a new problem. I figured it would only really be useful in cases where you knew what kind of physical problem you were working with already.
I can think of two examples for which ML is currently not very valuable but this new trick opens up. First, the blog post gave the example of Maxwell’s Equations—in EE there has been some exploration of things like genetic algorithms for circuit and antenna design, which boils down to a different form of searching the problem space for a solution. The hitch is, we can specify exactly what things to vary and what to keep constant; if we were to use ML it would always be starting from scratch, which is very inefficient. Until now, anyway.
The second one is also from engineering, but on the applied math side. The field of Continuum Mechanics has a certain generalization to thermodynamics called Rational Thermodynamics, which uses a method of moments. The idea there is simply to add another differential term to the field equation for every behavior you want to describe (heat, stress, deformation, etc). As of the 1980’s they had proved both that Navier Stokes is a special case and also that including many new terms does not significantly outperform Navier Stokes. The guess at that time was that it might take hundreds of thousands of terms to get a much better description, which of course was infeasible with analysis by hand. But having a lot of differentials to figure out seems exactly like what this would be good for.
Those are great examples! That’s exactly the sort of thing I see the tools currently associated with neural nets being most useful for long term—applications which aren’t really neural nets at all. Automated differentiation and optimization aren’t specific to neural nets, they’re generic mathematical tools. The neural network community just happens to be the main group developing them.
I really look forward to the day when I can bust out a standard DE solver, use it to estimate the frequency of some stable nonlinear oscillator, and then compute the sensitivity of that frequency to each of the DE’s parameters with an extra two lines of code.
It’s interesting that you mention that; I had never even considered applying the combination blindly to a new problem. I figured it would only really be useful in cases where you knew what kind of physical problem you were working with already.
I can think of two examples for which ML is currently not very valuable but this new trick opens up. First, the blog post gave the example of Maxwell’s Equations—in EE there has been some exploration of things like genetic algorithms for circuit and antenna design, which boils down to a different form of searching the problem space for a solution. The hitch is, we can specify exactly what things to vary and what to keep constant; if we were to use ML it would always be starting from scratch, which is very inefficient. Until now, anyway.
The second one is also from engineering, but on the applied math side. The field of Continuum Mechanics has a certain generalization to thermodynamics called Rational Thermodynamics, which uses a method of moments. The idea there is simply to add another differential term to the field equation for every behavior you want to describe (heat, stress, deformation, etc). As of the 1980’s they had proved both that Navier Stokes is a special case and also that including many new terms does not significantly outperform Navier Stokes. The guess at that time was that it might take hundreds of thousands of terms to get a much better description, which of course was infeasible with analysis by hand. But having a lot of differentials to figure out seems exactly like what this would be good for.
Those are great examples! That’s exactly the sort of thing I see the tools currently associated with neural nets being most useful for long term—applications which aren’t really neural nets at all. Automated differentiation and optimization aren’t specific to neural nets, they’re generic mathematical tools. The neural network community just happens to be the main group developing them.
I really look forward to the day when I can bust out a standard DE solver, use it to estimate the frequency of some stable nonlinear oscillator, and then compute the sensitivity of that frequency to each of the DE’s parameters with an extra two lines of code.