Re: “The key realization is that lossless data compression can only be achieved through empirical science. This is because data compression is impossible for arbitrary inputs: no compressor can ever achieve compression rates of less than M bits when averaged over all M-bit strings.”
Well, you only need one empirical observation to build a general purpose compressor—that inputs are more likely than not to be the result of executing short programs. Aftter noticing that you can obtain impressive compression ratios while ignoring the real world completely.
Aftter noticing that you can obtain impressive compression ratios while ignoring the real world completely.
Want to bet? :-)
An implicit point of my whole story is the rejection of the idea that good compression rates can be achieved using theory alone. I suppose you can disagree with this, but the history of science seems to be on my side—science never really started making progress until people started systematically checking their theories against empirical data.
In that case, you have compressed your “short programs” outputs. They are NOT unrelated to the real world as well, since you wrote them and you belong to the real world.
I didn’t say they were “related to the real world” in the first place, though.
What I mean is that you can build your Occam-based compressor without using any sensors that sample the world—outside of a closed computer programming domain which includes you and a computer.
Thomas, you do not seem to be doing “sympathetic reading”. As a result, you are ignoring what I am trying to say, and are instead focussing on irrelevant trivia. The result seems to be of pretty minimal interest in this case.
Re: “The key realization is that lossless data compression can only be achieved through empirical science. This is because data compression is impossible for arbitrary inputs: no compressor can ever achieve compression rates of less than M bits when averaged over all M-bit strings.”
Well, you only need one empirical observation to build a general purpose compressor—that inputs are more likely than not to be the result of executing short programs. Aftter noticing that you can obtain impressive compression ratios while ignoring the real world completely.
Want to bet? :-)
An implicit point of my whole story is the rejection of the idea that good compression rates can be achieved using theory alone. I suppose you can disagree with this, but the history of science seems to be on my side—science never really started making progress until people started systematically checking their theories against empirical data.
Why bet? This is a maths debate—and not a very complex one.
You don’t ignore the real world, if you compress the real world data well. At least you were lucky and guessed its physics.
If you haven’t, you can’t compress.
You don’t need to use real-word data in order to build your compressor. You can generate test data yourself using short programs.
In that case, you have compressed your “short programs” outputs. They are NOT unrelated to the real world as well, since you wrote them and you belong to the real world.
I didn’t say they were “related to the real world” in the first place, though.
What I mean is that you can build your Occam-based compressor without using any sensors that sample the world—outside of a closed computer programming domain which includes you and a computer.
It is NOT a closed domain. It is very much opened toward the real world. Since we belong there, we are a part of it.
Thomas, you do not seem to be doing “sympathetic reading”. As a result, you are ignoring what I am trying to say, and are instead focussing on irrelevant trivia. The result seems to be of pretty minimal interest in this case.