Well, I hope this post can be useful as a link you can give to explain the LW community’s mostly shared view about how one resolves the Problem of Induction. I wrote it because I think the LW Sequences’ treatment of the Problem of Induction is uncharacteristically off the mark.
I’m glad you wrote a post about this topic. When I was first reading the sequences, I didn’t find the posts by Eliezer on Induction very satisfying, and it was only after reading Jaynes and a bunch of papers on Solomonoff induction that I felt I had a better understanding of the situation. This post might have sped up that process for me by a day or two, if I had read it a year ago.
There was a little while where I thought Solomonoff Induction was a satisfying solution to the problem of induction. But there doesn’t seem to be any justification for the order over hypotheses in the Solomonoff Prior. Is there discussion/reading about this that I’m missing?
There are several related concepts (mostly from ML) that have caused me a lot of confusion, because of the way they overlap with each other and are often presented separately. These included Occam’s Razor and The Problem of Induction, and also “inductive bias”, “simplicity”, “generalisation”, overfitting, model bias and variance, and the general problem of assigning priors. I’d like there to be a post somewhere explaining the relationships between these words. I might try to write it, but I’m not confident I can make it clear.
Actually, I do appreciate you highlighting this, however, it’s because I think that Eliezer’s solution is somewhat underappreciated, which seems to be the opposite of what you think.
Well, I hope this post can be useful as a link you can give to explain the LW community’s mostly shared view about how one resolves the Problem of Induction. I wrote it because I think the LW Sequences’ treatment of the Problem of Induction is uncharacteristically off the mark.
I’m glad you wrote a post about this topic. When I was first reading the sequences, I didn’t find the posts by Eliezer on Induction very satisfying, and it was only after reading Jaynes and a bunch of papers on Solomonoff induction that I felt I had a better understanding of the situation. This post might have sped up that process for me by a day or two, if I had read it a year ago.
There was a little while where I thought Solomonoff Induction was a satisfying solution to the problem of induction. But there doesn’t seem to be any justification for the order over hypotheses in the Solomonoff Prior. Is there discussion/reading about this that I’m missing?
There are several related concepts (mostly from ML) that have caused me a lot of confusion, because of the way they overlap with each other and are often presented separately. These included Occam’s Razor and The Problem of Induction, and also “inductive bias”, “simplicity”, “generalisation”, overfitting, model bias and variance, and the general problem of assigning priors. I’d like there to be a post somewhere explaining the relationships between these words. I might try to write it, but I’m not confident I can make it clear.
Actually, I do appreciate you highlighting this, however, it’s because I think that Eliezer’s solution is somewhat underappreciated, which seems to be the opposite of what you think.
Ah yeah. Interesting how all the commenters here are talking about how this topic is quite obvious and settled, yet not saying the same things :)