Good point—I’ve struggled with the same problem, in different terms. Let me know if my statement of the problem matches the point you’re making here:
“It’s possible to discover, not just particulars about individual systems, but universal laws. These universal laws put a constraint on all future observations, thus reducing the subjective entropy of the universe, without (apparently) needing any corresponding gain of entropy.”
It’s something I was wondering about when going over the E. T. Jaynes papers and Yudkowsky’s Engines of Cognition.
I haven’t gotten it resolved in terms of 2nd law and the “subjective entropy” idea, but I think I know how to resolve it in the context of the simulated universe question: basically, if the simulation starts out adhering to the invariances that have to be obeyed (even though they might be more than necessary to fool observers), then it is no additional burden for the observers to notice these invariances.
Though the observers have (apparently) violated the 2nd law—and this is an area for further research—the simulator was already expending the computational resources necessary to make the invariances hold. It is an exception to the general principle I derived, in that it’s a case where net destruction of entropy requires no additional RAM.
I’m still working on how to resolve the remaining problems, but it shows how discovery of universal physical laws needn’t be a problem for the simulator.
I’ll try to bring your solution back to thermodynamics terms:
The universe always has and always will obey certain invariances, and those are a redundancy in your observations, which (along with any other redundancy that could possibly be derived) is already taken into account when computing information-theoretic entropy. If you had plenty of data already to derive the invariance but just hadn’t previously noticed it, that lack of logical omniscience is why the 2nd law is an inequality. Including the invariance into your future predictions isn’t a net reduction in entropy. It just removes some of the slack between the exact phase-volume preserving transforms of physics and the upper bounds that a computationally bounded agent has to use.
Your restatement looks exactly right, and your solution would resolve the issue I raised.
One question is, how much optimization can the simulators do if the true laws are as invariant as they “ought to be”? For example, if the universe has to evolve according to the same rules everywhere, that would seem to keep it from evolving in a chunkier way far away from us, which closes off a potential way to save on computation.
The simulator can maintain conservation of e.g. mass, while not churning through the computations required for e.g. gravity until people see enough that they can check if gravity isn’t holding.
This would save on having to do the gravity calculations. Then, when people, armed with their knowledge of gravity, start looking in more places, the universe must pick a configuration and stick with it—but at that point, all of their observations have the original problem of freeing up memory somewhere else in the form of higher entropy.
On second thought, that doesn’t work either, since discovery of gravitational laws will constrain their existing predictions of where the planets will be, and this destruction of entropy is unrelated to the entropy needed to create it, which was your objection to begin with.
My best guess at this point is that any resolution will ultimately hinge on a finer-grained information-theoretic analysis of the discovery of universal laws. That is, as you gain evidence pointing to the validity of laws you notice, you assign a high-but-not-unity probability to the laws continuing to hold. Each time your probability goes up, that corresponds to a particular reduction in the entropy of your probability distribution.
But, as they say, “to make inferences you have to make assumptions”. There is some entropic cost to making the assumptions necessary for the model with invariants to work, and this must be properly accounted for. I’ll continue to research this.
This would save on having to do the gravity calculations. Then, when people, armed with their knowledge of gravity, start looking in more places, the universe must pick a configuration and stick with it—but at that point, all of their observations have the original problem of freeing up memory somewhere else in the form of higher entropy.
This is wrong (even assuming that previous coarse-grained observations don’t matter). If you are changing the model by refining it, choosing one option of more detailed data arbitrarily, then this process on the world-model isn’t reversible: you can’t “un-choose” that arbitrary data and remain able to reconstruct it (unless the data is not arbitrary after all and only depends on the world model that is already there). As a result, no magical increase in entropy occurs, and no resources get saved: it’s not an operation on the subsystems within the modeled world, it’s an operation on the system of whole-world model within the world of modelers.
Also, consider the fact that ultimate laws can never be discovered, strictly speaking: there will always be uncertainty, and maybe there won’t even be asymptotically certain candidates, only turtles always deeper and deeper.
Good point—I’ve struggled with the same problem, in different terms. Let me know if my statement of the problem matches the point you’re making here:
“It’s possible to discover, not just particulars about individual systems, but universal laws. These universal laws put a constraint on all future observations, thus reducing the subjective entropy of the universe, without (apparently) needing any corresponding gain of entropy.”
It’s something I was wondering about when going over the E. T. Jaynes papers and Yudkowsky’s Engines of Cognition.
I haven’t gotten it resolved in terms of 2nd law and the “subjective entropy” idea, but I think I know how to resolve it in the context of the simulated universe question: basically, if the simulation starts out adhering to the invariances that have to be obeyed (even though they might be more than necessary to fool observers), then it is no additional burden for the observers to notice these invariances.
Though the observers have (apparently) violated the 2nd law—and this is an area for further research—the simulator was already expending the computational resources necessary to make the invariances hold. It is an exception to the general principle I derived, in that it’s a case where net destruction of entropy requires no additional RAM.
I’m still working on how to resolve the remaining problems, but it shows how discovery of universal physical laws needn’t be a problem for the simulator.
I’ll try to bring your solution back to thermodynamics terms:
The universe always has and always will obey certain invariances, and those are a redundancy in your observations, which (along with any other redundancy that could possibly be derived) is already taken into account when computing information-theoretic entropy. If you had plenty of data already to derive the invariance but just hadn’t previously noticed it, that lack of logical omniscience is why the 2nd law is an inequality. Including the invariance into your future predictions isn’t a net reduction in entropy. It just removes some of the slack between the exact phase-volume preserving transforms of physics and the upper bounds that a computationally bounded agent has to use.
Your restatement looks exactly right, and your solution would resolve the issue I raised.
One question is, how much optimization can the simulators do if the true laws are as invariant as they “ought to be”? For example, if the universe has to evolve according to the same rules everywhere, that would seem to keep it from evolving in a chunkier way far away from us, which closes off a potential way to save on computation.
The simulator can maintain conservation of e.g. mass, while not churning through the computations required for e.g. gravity until people see enough that they can check if gravity isn’t holding.
This would save on having to do the gravity calculations. Then, when people, armed with their knowledge of gravity, start looking in more places, the universe must pick a configuration and stick with it—but at that point, all of their observations have the original problem of freeing up memory somewhere else in the form of higher entropy.
On second thought, that doesn’t work either, since discovery of gravitational laws will constrain their existing predictions of where the planets will be, and this destruction of entropy is unrelated to the entropy needed to create it, which was your objection to begin with.
My best guess at this point is that any resolution will ultimately hinge on a finer-grained information-theoretic analysis of the discovery of universal laws. That is, as you gain evidence pointing to the validity of laws you notice, you assign a high-but-not-unity probability to the laws continuing to hold. Each time your probability goes up, that corresponds to a particular reduction in the entropy of your probability distribution.
But, as they say, “to make inferences you have to make assumptions”. There is some entropic cost to making the assumptions necessary for the model with invariants to work, and this must be properly accounted for. I’ll continue to research this.
This is wrong (even assuming that previous coarse-grained observations don’t matter). If you are changing the model by refining it, choosing one option of more detailed data arbitrarily, then this process on the world-model isn’t reversible: you can’t “un-choose” that arbitrary data and remain able to reconstruct it (unless the data is not arbitrary after all and only depends on the world model that is already there). As a result, no magical increase in entropy occurs, and no resources get saved: it’s not an operation on the subsystems within the modeled world, it’s an operation on the system of whole-world model within the world of modelers.
Also, consider the fact that ultimate laws can never be discovered, strictly speaking: there will always be uncertainty, and maybe there won’t even be asymptotically certain candidates, only turtles always deeper and deeper.