MML and KC are conceptually and theoretically highly related concepts, MML is another stab at formalizing Occam’s Razor in a more feasible manner, using the same approach as KC. No, they are not in fact identical, if that’s what you meant (hence the different names …)
Saying “my hypothesis has smaller Kolmogorov complexity than yours” is meaningless unless you can make the argument formal.
But saying “Based on Occam’s Razor, my hypothesis is smaller than yours” isn’t just as meaningless as long as your intuition stays sufficiently fuzzy and ungrounded? Is it an open problem as soon as anyone disagrees (or on what basis would you solve any dispute)? What use would the heuristic be, then?
I guess what I don’t understand is how you can embrace Occam’s Razor as an intuition, yet argue against the use of the branch of information theory that formalizes it, given there’s even computable variants. I agree that to categorically make statements about the KC of most hypotheses is misguided, and I also dislike the misuse of the terminology as mere buzzwords.
However, it is the formalism that our intuition is aspiring to emulate, and to improve our intuition would be to move it further towards the formalized basis it derives from, a move which you seem to reject.
But saying “Based on Occam’s Razor, my hypothesis is smaller than yours” isn’t just as meaningless as long as your intuition stays sufficiently fuzzy and ungrounded?
It’s not just a fuzzy intuition, you can try to count the concepts, but ultimately the argument remains informal. But throwing in “informal” Kolmogorov complexity doesn’t help, so what’s the point of doing that?
However, it is the formalism that our intuition is aspiring to emulate, and to improve our intuition would be to move it further towards the formalized basis it derives from, a move which you seem to reject.
I’m not sure that is the proper formalism, but even if it is, unless it provides actual tools to use in arguments, I think it’s not appropriate to use its terminology as buzzwords.
It’s not just a fuzzy intuition, you can try to count the concepts, but ultimately the argument remains informal.
Counting concepts is an error-prone, extremely rough approximation of complexity. A fuzzy, undependable version of it, if you will.
It falls to such problems such as (H1: A, B, C) versus (H2: A, D) with D being potentially larger or smaller than (B, C).
Or would you recommend trying to chunk out concepts of similar size? This will invariably lead you to the smallest differing unit, the smallest lexeme of your language of choice...
...and in the end, your “concept” will translate to “bit”, you’ll choose the the shortest equivalent restatement of the hypothesis with the fewest concepts (bits), and you’ll compare those. Familiar?
(t)[T]hrowing in “informal” Kolmogorov complexity doesn’t help, so what’s the point of doing that?
Think of it more as moving the intuition in the right direction. Of course that implies more than just usage of the terminology and precludes definitive statements (it’s still an intuition, not a formal calculation).
Such emphasis on the roots of our intuition can yield both positive and negative effects: Positive if used as a qualifier and a note of caution to our easily misguided “A is clearly more complex” intuitions, negative if we just append our intuition with “according to Kolmogorov Complexity” to lend unwarranted credence to our usual fallible guesstimating.
I’m not sure about what is exactly the focal point of our disagreement.
I’m not against making arguments more formal, I just don’t see how Kolmogorov complexity, Solomonoff induction, etc. can be practically used to that purpose.
MML and KC are conceptually and theoretically highly related concepts, MML is another stab at formalizing Occam’s Razor in a more feasible manner, using the same approach as KC. No, they are not in fact identical, if that’s what you meant (hence the different names …)
But saying “Based on Occam’s Razor, my hypothesis is smaller than yours” isn’t just as meaningless as long as your intuition stays sufficiently fuzzy and ungrounded? Is it an open problem as soon as anyone disagrees (or on what basis would you solve any dispute)? What use would the heuristic be, then?
I guess what I don’t understand is how you can embrace Occam’s Razor as an intuition, yet argue against the use of the branch of information theory that formalizes it, given there’s even computable variants. I agree that to categorically make statements about the KC of most hypotheses is misguided, and I also dislike the misuse of the terminology as mere buzzwords.
However, it is the formalism that our intuition is aspiring to emulate, and to improve our intuition would be to move it further towards the formalized basis it derives from, a move which you seem to reject.
It’s not just a fuzzy intuition, you can try to count the concepts, but ultimately the argument remains informal. But throwing in “informal” Kolmogorov complexity doesn’t help, so what’s the point of doing that?
I’m not sure that is the proper formalism, but even if it is, unless it provides actual tools to use in arguments, I think it’s not appropriate to use its terminology as buzzwords.
Counting concepts is an error-prone, extremely rough approximation of complexity. A fuzzy, undependable version of it, if you will.
It falls to such problems such as (H1: A, B, C) versus (H2: A, D) with D being potentially larger or smaller than (B, C).
Or would you recommend trying to chunk out concepts of similar size? This will invariably lead you to the smallest differing unit, the smallest lexeme of your language of choice...
...and in the end, your “concept” will translate to “bit”, you’ll choose the the shortest equivalent restatement of the hypothesis with the fewest concepts (bits), and you’ll compare those. Familiar?
Think of it more as moving the intuition in the right direction. Of course that implies more than just usage of the terminology and precludes definitive statements (it’s still an intuition, not a formal calculation).
Such emphasis on the roots of our intuition can yield both positive and negative effects: Positive if used as a qualifier and a note of caution to our easily misguided “A is clearly more complex” intuitions, negative if we just append our intuition with “according to Kolmogorov Complexity” to lend unwarranted credence to our usual fallible guesstimating.
I’m not sure about what is exactly the focal point of our disagreement.
I’m not against making arguments more formal, I just don’t see how Kolmogorov complexity, Solomonoff induction, etc. can be practically used to that purpose.