Right. We only assume uniformity for the same reason we assume all emeralds are green and not bleen. It’s just the simpler hypothesis. If we had reason to think that the laws of physics alternated like a checkerboard, or that colors magically changed in 2012, then we’d just have to take that into account.
This reminds me of the Feynman quote
“Philosophers say a great deal about what is absolutely necessary for science, and it is always, so far as one can see, rather naive, and probably wrong.”
I agree with Jimmy’s examples. Tim, the Solomonoff model may have some other fine print assumptions {see some analysis by Shane Legg here}, but “the earth having the same laws as space” or “laws not varying with time” are definitely not needed for the optimality proofs of the universal prior (though of course, to your point, uniformity does make our induction in practice easier, and time and space translation invariance of physical law do appear to be true, AFAIK.). Basically, assuming the universe is computable is enough to get the optimality guarantees. This doesn’t mean you might not still be wrong if Mars in empirical fact changes the rules you’ve learned on Earth, but it still provides a strong justification for using induction even if you were not guaranteed that the laws were the same, until you observed Mars to have different laws, at which point, you would assign largest weight to the simplest joint hypothesis for your next decision.
I’m afraid that you’re assuming what you’re trying to prove: whether you call it uniformity, or simplicity, or order, it’s all the same assumption, and you do have to assume it, whatever Feynman says.
Look at it from a Bayesian point of view: if your prior for the universe is that every sequence of Universe-states is equally likely, then the apparent order of the states so far gives no weight at all to more orderly future states—in fact, no observation can change what we expect.
Incidentally I’m very confident of the math in the paragraph above, and I’d ask that you’d be sure you’ve taken in what I’m getting at there in your reply.
There are a lot more complex than simple possible universes, so the assumption that an individual simple possible universe is more probable than an individual complex possible universe (which is the assumption being made here) is not the same thing as the assumption that all simple universes considered together are more probable than all complex universes considered together (i.e., the assumption that the universe is probably simple). (Not saying you disagree, but it’s probably good to be careful around the distinction.)
I suspect I’m going to be trying to make this point again at some point—I’ve had difficulty in the past explaining the problem of induction, and though I know about Solomonoff induction I only realised today that the whole problem is all about priors. I tried to to be explicit about which side of the distinction you draw I was speaking, but any thoughts on how I can make it clearer in future? Thanks!
Ciphergoth, I agree your points, that if your prior over world-states were not induction biased to start with, you would not be able to reliably use induction, and that this is a type of circularity. Also of course, the universe might just be such that the Occam prior doesn’t make you win; there is no free lunch, after all.
But I still think induction could meaningfully justify itself, at least in a partial sense. One possible, though speculative, pathway: Suppose Tegmark is right and all possible math structures exist, and that some of these contain conscious sub-structures, such as you. Suppose further that Bostrom is right and observers can be counted to constrain empirical predictions. Then it might be that there are more beings in your reference class that are part of simple mathematical structures as opposed to complex mathematical structures, possibly as a result of some mathematical fact about your structure and how that logically inter-relates to all possible structures. This might actually make something like induction true about the universe, without it needing to be a direct assumption. I personally don’t know if this will turn out to be true, nor whether it is provable even if true, but this would seem to me to be a deep, though still partially circular, justification for induction, if it is the case.
We’re not fully out of the woods even if all of this is true, because one still might want to ask Tegmark “Why does literally everything exist rather than something else?”, to which he might want to point to an Occam-like argument that “Everything exists” is algorithmically very simple. But these, while circularities, do not appear trivial to my mind; i.e., they are still deep and arguably meaningful connections which seem to lend credence to the whole edifice. Eli discusses in great detail why some circular loops like these might be ok/necessary to use in Where Recursive Justification Hits Bottom
Right. We only assume uniformity for the same reason we assume all emeralds are green and not bleen. It’s just the simpler hypothesis. If we had reason to think that the laws of physics alternated like a checkerboard, or that colors magically changed in 2012, then we’d just have to take that into account.
This reminds me of the Feynman quote “Philosophers say a great deal about what is absolutely necessary for science, and it is always, so far as one can see, rather naive, and probably wrong.”
I agree with Jimmy’s examples. Tim, the Solomonoff model may have some other fine print assumptions {see some analysis by Shane Legg here}, but “the earth having the same laws as space” or “laws not varying with time” are definitely not needed for the optimality proofs of the universal prior (though of course, to your point, uniformity does make our induction in practice easier, and time and space translation invariance of physical law do appear to be true, AFAIK.). Basically, assuming the universe is computable is enough to get the optimality guarantees. This doesn’t mean you might not still be wrong if Mars in empirical fact changes the rules you’ve learned on Earth, but it still provides a strong justification for using induction even if you were not guaranteed that the laws were the same, until you observed Mars to have different laws, at which point, you would assign largest weight to the simplest joint hypothesis for your next decision.
I’m afraid that you’re assuming what you’re trying to prove: whether you call it uniformity, or simplicity, or order, it’s all the same assumption, and you do have to assume it, whatever Feynman says.
Look at it from a Bayesian point of view: if your prior for the universe is that every sequence of Universe-states is equally likely, then the apparent order of the states so far gives no weight at all to more orderly future states—in fact, no observation can change what we expect.
Incidentally I’m very confident of the math in the paragraph above, and I’d ask that you’d be sure you’ve taken in what I’m getting at there in your reply.
There are a lot more complex than simple possible universes, so the assumption that an individual simple possible universe is more probable than an individual complex possible universe (which is the assumption being made here) is not the same thing as the assumption that all simple universes considered together are more probable than all complex universes considered together (i.e., the assumption that the universe is probably simple). (Not saying you disagree, but it’s probably good to be careful around the distinction.)
I suspect I’m going to be trying to make this point again at some point—I’ve had difficulty in the past explaining the problem of induction, and though I know about Solomonoff induction I only realised today that the whole problem is all about priors. I tried to to be explicit about which side of the distinction you draw I was speaking, but any thoughts on how I can make it clearer in future? Thanks!
Ciphergoth, I agree your points, that if your prior over world-states were not induction biased to start with, you would not be able to reliably use induction, and that this is a type of circularity. Also of course, the universe might just be such that the Occam prior doesn’t make you win; there is no free lunch, after all.
But I still think induction could meaningfully justify itself, at least in a partial sense. One possible, though speculative, pathway: Suppose Tegmark is right and all possible math structures exist, and that some of these contain conscious sub-structures, such as you. Suppose further that Bostrom is right and observers can be counted to constrain empirical predictions. Then it might be that there are more beings in your reference class that are part of simple mathematical structures as opposed to complex mathematical structures, possibly as a result of some mathematical fact about your structure and how that logically inter-relates to all possible structures. This might actually make something like induction true about the universe, without it needing to be a direct assumption. I personally don’t know if this will turn out to be true, nor whether it is provable even if true, but this would seem to me to be a deep, though still partially circular, justification for induction, if it is the case.
We’re not fully out of the woods even if all of this is true, because one still might want to ask Tegmark “Why does literally everything exist rather than something else?”, to which he might want to point to an Occam-like argument that “Everything exists” is algorithmically very simple. But these, while circularities, do not appear trivial to my mind; i.e., they are still deep and arguably meaningful connections which seem to lend credence to the whole edifice. Eli discusses in great detail why some circular loops like these might be ok/necessary to use in Where Recursive Justification Hits Bottom