(This is a basic point on conjunctions, but I don’t recall seeing its connection to Occam’s razor anywhere)
When I first read Occam’s Razorback in 2017, it seemed to me that the essay only addressed one kind of complexity: how complex the laws of physics are. If I’m not sure whether the witch did it, the universes where the witch did it are more complex, and so these explanations are exponentially less likely under a simplicity prior. Fine so far.
But there’s another type. Suppose I’m weighing whether the United States government is currently engaged in a vast conspiracy to get me to post this exact comment? This hypothesis doesn’t really demand a more complex source code, but I think we’d say that Occam’s razor shaves away this hypothesis anyways—even before weighing object-level considerations. This hypothesis is complex in a different way: it’s highly conjunctive in its unsupported claims about the current state of the world. Each conjunct eliminates many ways it could be true, from my current uncertainty, and so should I deem it correspondingly less likely.
I agree with the principle but I’m not sure I’d call it “Occam’s razor”. Occam’s razor is a bit sketchy, it’s not really a guarantee of anything, it’s not a mathematical law, it’s like a rule of thumb or something. Here you have a much more solid argument: multiplying many probabilities into a conjunction makes the result smaller and smaller. That’s a mathematical law, rock-solid. So I’d go with that...
My point was more that “people generally call both of these kinds of reasoning ‘Occam’s razor’, and they’re both good ways to reason, but they work differently.”
Oh, hmm, I guess that’s fair, now that you mention it I do recall hearing a talk where someone used “Occam’s razor” to talk about the solomonoff prior. Actually he called it “Bayes Occam’s razor” I think. He was talking about a probabilistic programming algorithm.
That’s (1) not physics, and (2) includes (as a special case) penalizing conjunctions, so maybe related to what you said. Or sorry if I’m still not getting what you meant
(This is a basic point on conjunctions, but I don’t recall seeing its connection to Occam’s razor anywhere)
When I first read Occam’s Razor back in 2017, it seemed to me that the essay only addressed one kind of complexity: how complex the laws of physics are. If I’m not sure whether the witch did it, the universes where the witch did it are more complex, and so these explanations are exponentially less likely under a simplicity prior. Fine so far.
But there’s another type. Suppose I’m weighing whether the United States government is currently engaged in a vast conspiracy to get me to post this exact comment? This hypothesis doesn’t really demand a more complex source code, but I think we’d say that Occam’s razor shaves away this hypothesis anyways—even before weighing object-level considerations. This hypothesis is complex in a different way: it’s highly conjunctive in its unsupported claims about the current state of the world. Each conjunct eliminates many ways it could be true, from my current uncertainty, and so should I deem it correspondingly less likely.
I agree with the principle but I’m not sure I’d call it “Occam’s razor”. Occam’s razor is a bit sketchy, it’s not really a guarantee of anything, it’s not a mathematical law, it’s like a rule of thumb or something. Here you have a much more solid argument: multiplying many probabilities into a conjunction makes the result smaller and smaller. That’s a mathematical law, rock-solid. So I’d go with that...
My point was more that “people generally call both of these kinds of reasoning ‘Occam’s razor’, and they’re both good ways to reason, but they work differently.”
Oh, hmm, I guess that’s fair, now that you mention it I do recall hearing a talk where someone used “Occam’s razor” to talk about the solomonoff prior. Actually he called it “Bayes Occam’s razor” I think. He was talking about a probabilistic programming algorithm.
That’s (1) not physics, and (2) includes (as a special case) penalizing conjunctions, so maybe related to what you said. Or sorry if I’m still not getting what you meant