>Occam’s razor. Is it saying anything other than P(A) >= P(A & B)?
Yes, this is the same as the argument for (the abstract importance of) Solomonoff Induction. (Though I guess you might not find it convincing.)
We have an intuitive sense that it’s simpler to say the world keeps existing when you turn your back on it. Likewise, it’s an intuitively simpler theory to say the laws of physics will continue to hold indefinitely, than to say the laws will hold up until February 12, 2023 at midnight Greenwich Mean Time. The law of probability which you cited only captures a small part of this idea, since it says that last theory has at least as much probability as the ‘simple’ version. We could add a rule saying nearly all of that probability accrues to the model where the laws keep holding, but what’s the general form of that rule?
Occam’s original formulation about not multiplying entities doesn’t help us much, as we could read this to say we shouldn’t assume the world keeps existing when unobserved. That’s the opposite of what we want. Newton’s version was arguably better, talking about “properties” which should be tentatively assumed to hold generally when we can’t find any evidence to the contrary, but then we could reasonably ask what “property” means.
SI comes from the idea that we should look for the minimum message length which predicts the data, and we could see SI in particular as an attempt to solve the grue-bleen problem. The “naturalized induction” school says this is technically still wrong, but it represents a major advance.
I’m not familiar with Solomonoff induction or minimum message length stuff either, sorry.
My first thought when I read the “world keeps existing” and “laws of physics keep holding” examples was to think that the conjunction rule covers it. Ie. something like P(these are the laws of physics) > P(these are the laws of physics & 2/12/23 and onwards things will be different). But I guess that begs the question of why. Why is the B a new condition rather than the default assumption, and I guess Occam’s razor is saying something like “Yes, it should be a new condition. The default assumption should be that the laws keep working”.
I’m struggling to understand how this generalizes though. Does it say that we should always assume that things will keep working the way they have been working?
I feel like that wouldn’t make sense. For the laws of physics we have evidence pointing towards them continuing to work. They worked yesterday, the day before that, the day before that, etc. But for new phenomena where we don’t have that evidence, I don’t see why we would make that assumption.
This probably isn’t a good example, but let’s say that ice cream was just recently invented and you are working as the first person taking orders. Your shop offers the flavors of chocolate, vanilla and strawberry. The first customer orders chocolate. Maybe Occam’s razor is saying that it’s simpler to assume that the next person will also order chocolate, and that people in general order chocolate? I think it’s more likely that they order chocolate than the other flavors, but we don’t have enough evidence to say that it’s significantly more likely.
And so more generally, what makes sense to me, is that how much we assume things will keep working the way they have been depends on the situation. Sometimes we have a lot of evidence that they will and so we feel confident that they will continue to. Other times we don’t.
I have a feeling at least some of what I said is wrong/misguided though.
Why do you need a distinction between new information and default assumptions? Why are default assumptions p=1?
If you know something, then assigning it p=1 makes sense...but assumption isn’t knowledge.
There’s a flaw in the conjunctive argument, in that assumes no knowledge of the propositions. Ten highly probable propositions can conjoin to a high probability, whereas two propositions with a default 0.5 each result in 0.25.
>Occam’s razor. Is it saying anything other than
P(A) >= P(A & B)?Yes, this is the same as the argument for (the abstract importance of) Solomonoff Induction. (Though I guess you might not find it convincing.)
We have an intuitive sense that it’s simpler to say the world keeps existing when you turn your back on it. Likewise, it’s an intuitively simpler theory to say the laws of physics will continue to hold indefinitely, than to say the laws will hold up until February 12, 2023 at midnight Greenwich Mean Time. The law of probability which you cited only captures a small part of this idea, since it says that last theory has at least as much probability as the ‘simple’ version. We could add a rule saying nearly all of that probability accrues to the model where the laws keep holding, but what’s the general form of that rule?
Occam’s original formulation about not multiplying entities doesn’t help us much, as we could read this to say we shouldn’t assume the world keeps existing when unobserved. That’s the opposite of what we want. Newton’s version was arguably better, talking about “properties” which should be tentatively assumed to hold generally when we can’t find any evidence to the contrary, but then we could reasonably ask what “property” means.
SI comes from the idea that we should look for the minimum message length which predicts the data, and we could see SI in particular as an attempt to solve the grue-bleen problem. The “naturalized induction” school says this is technically still wrong, but it represents a major advance.
I’m not familiar with Solomonoff induction or minimum message length stuff either, sorry.
My first thought when I read the “world keeps existing” and “laws of physics keep holding” examples was to think that the conjunction rule covers it. Ie. something like
P(these are the laws of physics) > P(these are the laws of physics & 2/12/23 and onwards things will be different). But I guess that begs the question of why. Why is the B a new condition rather than the default assumption, and I guess Occam’s razor is saying something like “Yes, it should be a new condition. The default assumption should be that the laws keep working”.I’m struggling to understand how this generalizes though. Does it say that we should always assume that things will keep working the way they have been working?
I feel like that wouldn’t make sense. For the laws of physics we have evidence pointing towards them continuing to work. They worked yesterday, the day before that, the day before that, etc. But for new phenomena where we don’t have that evidence, I don’t see why we would make that assumption.
This probably isn’t a good example, but let’s say that ice cream was just recently invented and you are working as the first person taking orders. Your shop offers the flavors of chocolate, vanilla and strawberry. The first customer orders chocolate. Maybe Occam’s razor is saying that it’s simpler to assume that the next person will also order chocolate, and that people in general order chocolate? I think it’s more likely that they order chocolate than the other flavors, but we don’t have enough evidence to say that it’s significantly more likely.
And so more generally, what makes sense to me, is that how much we assume things will keep working the way they have been depends on the situation. Sometimes we have a lot of evidence that they will and so we feel confident that they will continue to. Other times we don’t.
I have a feeling at least some of what I said is wrong/misguided though.
While it’s arguably good for you to understand the confusion which led to it, you might want to actually just look up Solomonoff Induction now.
Solomonoff Induction has always felt a little intimidating but I see how it’s relevant so yeah, I’ll check it out at some point.
Why do you need a distinction between new information and default assumptions? Why are default assumptions p=1?
If you know something, then assigning it p=1 makes sense...but assumption isn’t knowledge.
There’s a flaw in the conjunctive argument, in that assumes no knowledge of the propositions. Ten highly probable propositions can conjoin to a high probability, whereas two propositions with a default 0.5 each result in 0.25.
But intuitions are just subjective flapdoodle, aren’t they? ;-)