But I’m still confused as to whether you’re claiming that someone could implement AIXI and feed it all the data you mentioned.
That was a joke, but computable approximations of AIXI can certainly be implemented. For instance, a logical inductor run on all that data would be conceptually similar for our purposes.
You seem to be claiming that “Wikipedia” (or all of the scientific data ever measured) would be enough to generate “the prior distribution over positions and momenta of every particle” and that this data would easily fit on a hard drive.
No, wikipedia or a bunch of scientific data (much less than all the scientific data ever measured), would be enough data to train a solid climate model from a simple prior over particle distributions and momenta. It would definitely not be enough to learn the position and momentum of every particle; a key point of stat mech is that we do not need to learn the position and momentum of every particle in order to make macroscopic predictions. A simple maxentropic prior over microscopic states plus a (relatively) small amount of macroscopic data is enough to make macroscopic predictions.
This makes me believe that you’re referring to some kind of theoretical algorithm.
The code itself need not be theoretical, but it would definitely be superexponentially slow to run. Making it efficient is where stat mech, multiscale modelling, etc come in. The point I want to make is that the system’s “complexity” is not a fundamental barrier requiring fundamentally different epistemic principles.
… wikipedia or a bunch of scientific data (much less than all the scientific data ever measured), would be enough data to train a solid climate model from a simple prior over particle distributions and momenta. It would definitely not be enough to learn the position and momentum of every particle; a key point of stat mech is that we do not need to learn the position and momentum of every particle in order to make macroscopic predictions. A simple maxentropic prior over microscopic states plus a (relatively) small amount of macroscopic data is enough to make macroscopic predictions.
That’s clearer to me, but I’m still skeptical that that’s in fact possible. I don’t understand how the prior can be considered “over particle distributions and momenta”, except via the theories and models of statistical mechanics, i.e. assuming that those microscopic details can be ignored.
The point I want to make is that the system’s “complexity” is not a fundamental barrier requiring fundamentally different epistemic principles.
I agree with this. But I think you’re eliding how much work is involved in what you described as:
Making it efficient is where stat mech, multiscale modelling, etc come in.
I wouldn’t think that standard statistical mechanics would be sufficient for modeling the Earth’s climate. I’d expect fluid dynamics is also important as well as chemistry, geology, the dynamics of the Sun, etc.. It’s not obvious to me that statistical mechanics would be effective alone in practice.
Ah… I’m talking about stat mech in a broader sense than I think you’re imagining. The central problem of the field is the “bridge laws” defining/expressing macroscopic behavior in terms of microscopic behavior. So, e.g., deriving Navier-Stokes from molecular dynamics is a stat mech problem. Of course we still need the other sciences (chemistry, geology, etc) to define the system in the first place. The point of stat mech is to take low-level laws with lots of degrees of freedom, and derive macroscopic laws from them. For very coarse, high-level models, the “low-level model” might itself be e.g. fluid dynamics.
I think you’re eliding how much work is involved in what you described as...
Yeah, this stuff definitely isn’t easy. As you argued above, the general case of the problem is basically AGI (and also the topic of my own research). But there are a lot of existing tricks and the occasional reasonably-general-tool, especially in the multiscale modelling world and in Bayesian stat mech.
Yes, I don’t think we really disagree. My prior (prior to this extended comments discussion) was that there are lots of wonderful existing tricks, but there’s no real shortcut for the fully general problem and any such shortcut would be effectively AGI anyways.
That was a joke, but computable approximations of AIXI can certainly be implemented. For instance, a logical inductor run on all that data would be conceptually similar for our purposes.
No, wikipedia or a bunch of scientific data (much less than all the scientific data ever measured), would be enough data to train a solid climate model from a simple prior over particle distributions and momenta. It would definitely not be enough to learn the position and momentum of every particle; a key point of stat mech is that we do not need to learn the position and momentum of every particle in order to make macroscopic predictions. A simple maxentropic prior over microscopic states plus a (relatively) small amount of macroscopic data is enough to make macroscopic predictions.
The code itself need not be theoretical, but it would definitely be superexponentially slow to run. Making it efficient is where stat mech, multiscale modelling, etc come in. The point I want to make is that the system’s “complexity” is not a fundamental barrier requiring fundamentally different epistemic principles.
That’s clearer to me, but I’m still skeptical that that’s in fact possible. I don’t understand how the prior can be considered “over particle distributions and momenta”, except via the theories and models of statistical mechanics, i.e. assuming that those microscopic details can be ignored.
I agree with this. But I think you’re eliding how much work is involved in what you described as:
I wouldn’t think that standard statistical mechanics would be sufficient for modeling the Earth’s climate. I’d expect fluid dynamics is also important as well as chemistry, geology, the dynamics of the Sun, etc.. It’s not obvious to me that statistical mechanics would be effective alone in practice.
Ah… I’m talking about stat mech in a broader sense than I think you’re imagining. The central problem of the field is the “bridge laws” defining/expressing macroscopic behavior in terms of microscopic behavior. So, e.g., deriving Navier-Stokes from molecular dynamics is a stat mech problem. Of course we still need the other sciences (chemistry, geology, etc) to define the system in the first place. The point of stat mech is to take low-level laws with lots of degrees of freedom, and derive macroscopic laws from them. For very coarse, high-level models, the “low-level model” might itself be e.g. fluid dynamics.
Yeah, this stuff definitely isn’t easy. As you argued above, the general case of the problem is basically AGI (and also the topic of my own research). But there are a lot of existing tricks and the occasional reasonably-general-tool, especially in the multiscale modelling world and in Bayesian stat mech.
Yes, I don’t think we really disagree. My prior (prior to this extended comments discussion) was that there are lots of wonderful existing tricks, but there’s no real shortcut for the fully general problem and any such shortcut would be effectively AGI anyways.