Without induction, you cannot update on evidence, because “evidence” is a hollow concept.
Still wrong. As I pointed out in the main post, “The sun will rise tomorrow” has a probability of 1.0 given that “The sun will rise for the next 10,000 days.” This means that the sun rising tomorrow is evidence that the sun will rise for the next 10,000 days, without presupposing induction.
For example, I might originally be convinced that the probability of the sun rising tomorrow is one in a billion, and the probability of it rising for the next 10,000 days, one in a google; i.e. I am convinced that induction is wrong and the future will not be like the past. Nonetheless Bayes’ theorem inexorably forces me to update in favor of the sun rising for the next 10,000 days, if it rises tomorrow.
Wait, what? “The sun will rise tomorrow” also has a probability of 1.0 given that “The sun will rise tomorrow and then never rise again”, so the sun rising tomorrow should make you update in favor of that hypothesis too. Why did you choose to focus on a hypothesis saying the future (starting from the day after tomorrow) will be like the past (tomorrow)? This is circular—the problem of induction all over again.
Yes, the probability of that will also increase after the first day—it is perfectly consistent for the probability of both hypotheses to increase. But the day after that, the probability of the sun rising the next 10,000 days has increased even more, and the probability of your hypothesis has dropped to zero.
As I said in the post, people do in fact formulate universal hypotheses, namely ones which will suggest that the future is like the past. But you don’t have to assume the future will be like the past to do this; as I said in the previous comment, you might even be assuming that the future will NOT be like the past. The Bayesian reasoning will work just the same.
But the day after that, the probability of the sun rising the next 10,000 days has increased even more, and the probability of your hypothesis has dropped to zero.
Who says this will happen in the first place? Even if you personally know that the sun will rise tomorrow like it always did, you’re not allowed to use that fact while solving the problem of induction.
Maybe an important insight into why the justification of induction remains so puzzling. We should be justifying induction as a policy, not as a magic-bullet formula which works in each and every instance.
Oh, and not to be nitpicky, but if you’re relying on any kind of metric (e.g. your vision) to ascertain that the sun does rise tomorrow, you rely on induction. Without induction, there is simply no way of establishing that your observations correlate with anything meaningful. “The sun will rise tomorrow” cannot actually be confirmed without assuming induction; without the evidence confirming their reliability from past experience, our sensory data are meaningless. This is getting into a nightmarish level of abstraction for a relatively simple point, though.
Believing the sun will rise tomorrow with P=10^-9 is not failing to believe in induction. It’s making a serious mistake, or being privy to some very interesting evidence. Without induction, no probability estimate is possible, because we simply have no idea what will happen.
I suspect this argument stems from different definitions of “induction.”
If you define believing in induction as believing that the future will be like the past, it is possible to believe that the future will not be like the past, and one example of that would be believing that the sun will not rise tomorrow. Similarly, someone could suppose that everything that will happen tomorrow will be totally different from today, and he could still use Bayes’ theorem, if he had any probability estimates at all.
You say, “Without induction, no probability estimate is possible, because we simply have no idea what will happen.” Probability estimates are subjective degrees of belief, and it may be that there is some process like induction that generates them in a person’s mind. But this doesn’t mean that he believes, intellectually, that the future will be like the past, nor that he actually uses this claim in coming up with an estimate; as I just pointed out, some claims explicitly deny that induction will continue to work, and some people sometimes believe them (i.e. “The world will end tomorrow!”)
In any case, it doesn’t matter how a person comes up with his subjective estimates; a prior probability estimate doesn’t need to be justified, it just needs to be used. This post was not intended to justify people’s priors, but the process of induction as an explicit reasoning process—which is not used in generating priors.
I suspect that the argument arises because, deep down, you don’t yet accept that Bayes theorem is more fundamental than induction and that it shows us how to use evidence other than inductive evidence.
That said, you may well be correct in your “nitpick” to the effect that we wouldn’t even be able to interpret sense data as ordinary everyday evidence without induction. That may well be, which would mean that we have to use induction and Bayes theorem at the sense data level before we can use Bayes at the ordinary everyday evidence level. But that does not make induction as fundamental as Bayes.
Since my original point was amended to indicate that my original point about Bayes was overstated, and that the true problem is that Bayes is quite useless without assuming induction is justified (i.e. any observation about the real world or prediction about the future presumes the principle of induction to be justified), I would hardly call this nitpicking. It is my point. Insofar as Bayes’ theorem is purely mathematical, it is quite fundamental. I don’t dispute that. You can’t apply math to the real world without having a real world, and without assuming induction, you can’t really have a concept of a real world.
It has occurred to me that the concept of “induction” upon which I rely may be different in nature from that being used by the people arguing with me. This is unsurprisingly causing problems. Induction, as I mean it, is not simply, “the future will be like the past,” but, “the correlation between past observations and future observations is nonzero.” That is so fundamental I do not think the human mind is capable of not essentially believing it.
If induction means “the correlation between past observations and future observations in nonzero,” then not assuming induction could mean one of two things:
1)I might think there is some chance that the correlation is non-zero, and some chance that the correlation is zero. In this case Bayesian reasoning will still work, and confirms that the correlation is non-zero.
2) I might think the correlation is certainly zero. But in this case most people would not describe this as “not assuming induction”, but as making a completely unjustified and false assumption instead. It is not negative (not assuming) but positive (assuming something.)
A universe in which every kind of past observation is uncorrelated with future observations of the same kind would be a world in which animals could not evolve. Hence, not the kind of universe I would care to (or be able to) contemplate. However, I am quite capable of believing that there are some kinds of observations which do not correlate with future instances of themselves. Random noise exists.
Assuming you have no objection to that, I suppose you can go on preaching that the key mystery of the universe and the basis of all epistemology is induction. I have no objection. But I do think you ought to read Jaynes. Who knows? You might find something there to change your mind or perhaps a clue to dissolving the mystery.
I don’t think induction is of particular importance. We can’t function without assuming its validity. Thus, entertaining the idea that it is invalid is not constructive. I’d be very curious to see someone solve the problem of induction (which I briefly thought this was an attempt at), but it’s hardly an urgent matter.
Picking up on animals not evolving makes about as much sense as picking up on the fact that, if it weren’t for gravity, it would be tough to play badminton. This reinforces my suspicion that our concept of what we’re arguing about is so vastly different that a productive resolution is impossible.
I suppose the origin of this whole digression could be summarized by saying I thought the post was about (the problem of) induction, and was a useless point about a moderately interesting topic. Instead, it’s about (the practice of) induction, making it a decent but not terribly useful point about a rather uninteresting (or at least simple) topic. It is perhaps even less salient than the observation that, if we assume infinite sets of possibilities, then at some point Occam’s razor must work by sheer force of the nature of finite sum infinite sets having to have some arbitrary point after which they decrease.
It is perhaps even less salient than the observation that, if we assume infinite sets of possibilities, then at some point Occam’s razor must work by sheer force of the nature of finite sum infinite sets having to have some arbitrary point after which they decrease.
“The sun will rise tomorrow” has a probability of 1.0 given that “The sun will rise for the next 10,000 days.”
True. Not a counter example. Not an example, actually. There is no evidence in, “the sun will rise tomorrow if the sun will rise every day for the next 10,000 years.” That statement derives its truth from the meaning of the words. It would be equally true if there were no sun and there were no tomorrow. Furthermore, without induction, you can never arrive at a probability estimate of the sun rising tomorrow in the first place. With no such thing as evidence, there’s simply no way to construct a probability estimate. Your examples are assuming themselves past the exact problem I’m trying to point out.
A world where induction ceased to hold up is literally unimaginable. I don’t think a sane (or even insane) person could conceive of a world in which induction does not work and the future does not have any relation to the past. That is likely why the problem is so difficult to understand.
I don’t think you know how Bayes’ theorem works… if I say “A & B will both be true,” and it turns out that A is true, this is evidence for my original claim, despite the fact that it is implied by the meaning of the words… or rather in this case, because it is so implied.
Also, without induction, we can construct a probability estimate: take all the possibilities we know of, add the possibility “or something else”, and then say that each of possibilities is equally likely. Yes, this probability estimate isn’t likely to be calibrated but it will be the best we can do given the condition of not knowing anything about the relation of past and future.
How can one confirm that this happens without induction?
That said, I’m becoming somewhat more convinced of your point; I’m just not sure it’s of any practical value.
The issue is that any statement made about the real world requires induction. If we are speaking in the hypothetical, yes, if the sun rises tomorrow, then that increases the chance that the sun will rise for the next ten thousand days, in the same sense that if the next flozit we quivel is bzorgy, that increases the chance that the next 10,000 flozits we quivel will be bzorgy. It tells us nothing new or interesting about the actual world. Furthermore, we can never confirm that the conditional is fulfilled without sense data, the reliability of which presumes induction.
Just as you seem to think I underestimate the primacy of Bayes, I think you significantly underrate the depth of induction. A world without induction being valid is not so much one where the sun does not rise tomorrow as one that would make a drugged-out madman look like a Bayesian superintelligence. Without assuming the validity of induction, that there will be an earth or a sun or a tomorrow are all completely uncertain propositions. Without induction, far more is possible than we can ever hope to imagine. We can’t construct any meaningful probability estimate, because without induction we would expect the world to look something like more colorful TV static.
The fact that your senses are reliable may “presume induction”, just as it may also presume that you are not being deceived by Descartes’ evil demon. But when you say “that grass is green,” you don’t consider that the only reason you know it isn’t red is because there isn’t any demon… instead you don’t think of Descartes’ demon at all, and likewise you don’t think of induction at all. In any case, in whatever way “induction” might be involved in your senses, my article doesn’t intend to consider this, but induction considered just as a way of reasoning.
My point is that a year ago, you could have had a 90% probability estimate that the world would look like TV static, and a 10% probability of something else. But the 10% chance has been confirmed, not the 90% chance. Or if you say that there was no basis for the 10% chance, then maybe you thought there was a 100% chance of static. But in this case you collapse in Bayesian explosion, since the static didn’t happen.
In other words, “not assuming induction” does not mean being 100% certain that there will not be a future or that it will not be like the past; it means being uncertain, which means having degrees of belief.
Still wrong. As I pointed out in the main post, “The sun will rise tomorrow” has a probability of 1.0 given that “The sun will rise for the next 10,000 days.” This means that the sun rising tomorrow is evidence that the sun will rise for the next 10,000 days, without presupposing induction.
Right. Induction only comes in when I infer, from the sun’s rising tomorrow, that it will rise on the 9,999 days after tomorrow.
Replying to your edit:
Still wrong. As I pointed out in the main post, “The sun will rise tomorrow” has a probability of 1.0 given that “The sun will rise for the next 10,000 days.” This means that the sun rising tomorrow is evidence that the sun will rise for the next 10,000 days, without presupposing induction.
For example, I might originally be convinced that the probability of the sun rising tomorrow is one in a billion, and the probability of it rising for the next 10,000 days, one in a google; i.e. I am convinced that induction is wrong and the future will not be like the past. Nonetheless Bayes’ theorem inexorably forces me to update in favor of the sun rising for the next 10,000 days, if it rises tomorrow.
Wait, what? “The sun will rise tomorrow” also has a probability of 1.0 given that “The sun will rise tomorrow and then never rise again”, so the sun rising tomorrow should make you update in favor of that hypothesis too. Why did you choose to focus on a hypothesis saying the future (starting from the day after tomorrow) will be like the past (tomorrow)? This is circular—the problem of induction all over again.
Yes, the probability of that will also increase after the first day—it is perfectly consistent for the probability of both hypotheses to increase. But the day after that, the probability of the sun rising the next 10,000 days has increased even more, and the probability of your hypothesis has dropped to zero.
As I said in the post, people do in fact formulate universal hypotheses, namely ones which will suggest that the future is like the past. But you don’t have to assume the future will be like the past to do this; as I said in the previous comment, you might even be assuming that the future will NOT be like the past. The Bayesian reasoning will work just the same.
Who says this will happen in the first place? Even if you personally know that the sun will rise tomorrow like it always did, you’re not allowed to use that fact while solving the problem of induction.
The only reason that I’m using it is because in real life we update more than once.
But if you want to focus on the single update, yes, both hypotheses become more probable.
Maybe an important insight into why the justification of induction remains so puzzling. We should be justifying induction as a policy, not as a magic-bullet formula which works in each and every instance.
Oh, and not to be nitpicky, but if you’re relying on any kind of metric (e.g. your vision) to ascertain that the sun does rise tomorrow, you rely on induction. Without induction, there is simply no way of establishing that your observations correlate with anything meaningful. “The sun will rise tomorrow” cannot actually be confirmed without assuming induction; without the evidence confirming their reliability from past experience, our sensory data are meaningless. This is getting into a nightmarish level of abstraction for a relatively simple point, though.
Believing the sun will rise tomorrow with P=10^-9 is not failing to believe in induction. It’s making a serious mistake, or being privy to some very interesting evidence. Without induction, no probability estimate is possible, because we simply have no idea what will happen.
I suspect this argument stems from different definitions of “induction.”
If you define believing in induction as believing that the future will be like the past, it is possible to believe that the future will not be like the past, and one example of that would be believing that the sun will not rise tomorrow. Similarly, someone could suppose that everything that will happen tomorrow will be totally different from today, and he could still use Bayes’ theorem, if he had any probability estimates at all.
You say, “Without induction, no probability estimate is possible, because we simply have no idea what will happen.” Probability estimates are subjective degrees of belief, and it may be that there is some process like induction that generates them in a person’s mind. But this doesn’t mean that he believes, intellectually, that the future will be like the past, nor that he actually uses this claim in coming up with an estimate; as I just pointed out, some claims explicitly deny that induction will continue to work, and some people sometimes believe them (i.e. “The world will end tomorrow!”)
In any case, it doesn’t matter how a person comes up with his subjective estimates; a prior probability estimate doesn’t need to be justified, it just needs to be used. This post was not intended to justify people’s priors, but the process of induction as an explicit reasoning process—which is not used in generating priors.
I suspect that the argument arises because, deep down, you don’t yet accept that Bayes theorem is more fundamental than induction and that it shows us how to use evidence other than inductive evidence.
That said, you may well be correct in your “nitpick” to the effect that we wouldn’t even be able to interpret sense data as ordinary everyday evidence without induction. That may well be, which would mean that we have to use induction and Bayes theorem at the sense data level before we can use Bayes at the ordinary everyday evidence level. But that does not make induction as fundamental as Bayes.
Since my original point was amended to indicate that my original point about Bayes was overstated, and that the true problem is that Bayes is quite useless without assuming induction is justified (i.e. any observation about the real world or prediction about the future presumes the principle of induction to be justified), I would hardly call this nitpicking. It is my point. Insofar as Bayes’ theorem is purely mathematical, it is quite fundamental. I don’t dispute that. You can’t apply math to the real world without having a real world, and without assuming induction, you can’t really have a concept of a real world.
It has occurred to me that the concept of “induction” upon which I rely may be different in nature from that being used by the people arguing with me. This is unsurprisingly causing problems. Induction, as I mean it, is not simply, “the future will be like the past,” but, “the correlation between past observations and future observations is nonzero.” That is so fundamental I do not think the human mind is capable of not essentially believing it.
If induction means “the correlation between past observations and future observations in nonzero,” then not assuming induction could mean one of two things:
1)I might think there is some chance that the correlation is non-zero, and some chance that the correlation is zero. In this case Bayesian reasoning will still work, and confirms that the correlation is non-zero.
2) I might think the correlation is certainly zero. But in this case most people would not describe this as “not assuming induction”, but as making a completely unjustified and false assumption instead. It is not negative (not assuming) but positive (assuming something.)
A universe in which every kind of past observation is uncorrelated with future observations of the same kind would be a world in which animals could not evolve. Hence, not the kind of universe I would care to (or be able to) contemplate. However, I am quite capable of believing that there are some kinds of observations which do not correlate with future instances of themselves. Random noise exists.
Assuming you have no objection to that, I suppose you can go on preaching that the key mystery of the universe and the basis of all epistemology is induction. I have no objection. But I do think you ought to read Jaynes. Who knows? You might find something there to change your mind or perhaps a clue to dissolving the mystery.
[Edit: Removed opening snark.]
I don’t think induction is of particular importance. We can’t function without assuming its validity. Thus, entertaining the idea that it is invalid is not constructive. I’d be very curious to see someone solve the problem of induction (which I briefly thought this was an attempt at), but it’s hardly an urgent matter.
Picking up on animals not evolving makes about as much sense as picking up on the fact that, if it weren’t for gravity, it would be tough to play badminton. This reinforces my suspicion that our concept of what we’re arguing about is so vastly different that a productive resolution is impossible.
I suppose the origin of this whole digression could be summarized by saying I thought the post was about (the problem of) induction, and was a useless point about a moderately interesting topic. Instead, it’s about (the practice of) induction, making it a decent but not terribly useful point about a rather uninteresting (or at least simple) topic. It is perhaps even less salient than the observation that, if we assume infinite sets of possibilities, then at some point Occam’s razor must work by sheer force of the nature of finite sum infinite sets having to have some arbitrary point after which they decrease.
Ouch. Burn
True. Not a counter example. Not an example, actually. There is no evidence in, “the sun will rise tomorrow if the sun will rise every day for the next 10,000 years.” That statement derives its truth from the meaning of the words. It would be equally true if there were no sun and there were no tomorrow. Furthermore, without induction, you can never arrive at a probability estimate of the sun rising tomorrow in the first place. With no such thing as evidence, there’s simply no way to construct a probability estimate. Your examples are assuming themselves past the exact problem I’m trying to point out.
A world where induction ceased to hold up is literally unimaginable. I don’t think a sane (or even insane) person could conceive of a world in which induction does not work and the future does not have any relation to the past. That is likely why the problem is so difficult to understand.
I don’t think you know how Bayes’ theorem works… if I say “A & B will both be true,” and it turns out that A is true, this is evidence for my original claim, despite the fact that it is implied by the meaning of the words… or rather in this case, because it is so implied.
Also, without induction, we can construct a probability estimate: take all the possibilities we know of, add the possibility “or something else”, and then say that each of possibilities is equally likely. Yes, this probability estimate isn’t likely to be calibrated but it will be the best we can do given the condition of not knowing anything about the relation of past and future.
How can one confirm that this happens without induction?
That said, I’m becoming somewhat more convinced of your point; I’m just not sure it’s of any practical value.
The issue is that any statement made about the real world requires induction. If we are speaking in the hypothetical, yes, if the sun rises tomorrow, then that increases the chance that the sun will rise for the next ten thousand days, in the same sense that if the next flozit we quivel is bzorgy, that increases the chance that the next 10,000 flozits we quivel will be bzorgy. It tells us nothing new or interesting about the actual world. Furthermore, we can never confirm that the conditional is fulfilled without sense data, the reliability of which presumes induction.
Just as you seem to think I underestimate the primacy of Bayes, I think you significantly underrate the depth of induction. A world without induction being valid is not so much one where the sun does not rise tomorrow as one that would make a drugged-out madman look like a Bayesian superintelligence. Without assuming the validity of induction, that there will be an earth or a sun or a tomorrow are all completely uncertain propositions. Without induction, far more is possible than we can ever hope to imagine. We can’t construct any meaningful probability estimate, because without induction we would expect the world to look something like more colorful TV static.
The fact that your senses are reliable may “presume induction”, just as it may also presume that you are not being deceived by Descartes’ evil demon. But when you say “that grass is green,” you don’t consider that the only reason you know it isn’t red is because there isn’t any demon… instead you don’t think of Descartes’ demon at all, and likewise you don’t think of induction at all. In any case, in whatever way “induction” might be involved in your senses, my article doesn’t intend to consider this, but induction considered just as a way of reasoning.
My point is that a year ago, you could have had a 90% probability estimate that the world would look like TV static, and a 10% probability of something else. But the 10% chance has been confirmed, not the 90% chance. Or if you say that there was no basis for the 10% chance, then maybe you thought there was a 100% chance of static. But in this case you collapse in Bayesian explosion, since the static didn’t happen.
In other words, “not assuming induction” does not mean being 100% certain that there will not be a future or that it will not be like the past; it means being uncertain, which means having degrees of belief.
Right. Induction only comes in when I infer, from the sun’s rising tomorrow, that it will rise on the 9,999 days after tomorrow.