Re: “Attempts to reduce intelligence to comression butt up against the same limits that compression does: you can be good at compressing some kinds of data, only if you sacrifice ability to compress other kinds of data.”
That is not a meaningful limitation. There are general purpose universal compressors. It is part of the structure of reality that sequences generated by short programs are more commonly observed. That’s part of the point of using a compressor—it is an automated way of applying Occam’s razor.
That is not a meaningful limitation. There are general purpose universal compressors.
There are frequently useful general purpose compressors that work by anticipating the most common regularities in the set of files typically generated by humans. But they do not, and cannot, iterate through all the short programs that could have generated the data—it’s too time-consuming.
The point was that general purpose compression is possible. Yes, you sacrifice the ability to compress other kinds of data—but those other kinds of data are highly incompressible and close to random—not the kind of data which most intelligent agents are interested in finding patterns in in the first place.
Yes, you sacrifice the ability to compress other kinds of data—but those other kinds of data are highly incompressible and close to random.
No, they look random and incompressible because effective compression algorithms optimized for this universe can’t compress them. But algorithms optimized for other computable universes may regard them as normal and have a good way to compress them.
Which kinds of data (from computable processes) are likely to be observed in this universe? Ay, there’s the rub.
Compressing sequences from this universe is good enough for me.
Except that the problem you were attacking at the beginning of this thread was general intelligence, which you claimed to be solvable just by good enough compression, but that requires knowing which parts of the search space in this universe are unlikely, which you haven’t shown how to algorithmize.
“Which kinds of data (from computable processes) are likely to be observed in this universe? Ay, there’s the rub.”
Not really—there are well-known results about that—see: …
Yes, but as I keep trying to say, those results are far from enough to get something workable, and it’s not the methodology behind general compression programs.
Arithmetic compression, Huffman compression, Lempel-Ziv compression, etc are all excellent at compressing sequences produced by small programs. Things like:
Those compressors (crudely) implement a computable approximation of Solomonoff induction without iterating through programs that generate the output. How they work is not very relevant here—the point is that they act as general-purpose compressors—and compress a great range of real world data types.
The complaint that we don’t know what types of data are in the universe is just not applicable—we do, in fact, know a considerable amount about that—and that is why we can build general purpose compressors.
Re: “Attempts to reduce intelligence to comression butt up against the same limits that compression does: you can be good at compressing some kinds of data, only if you sacrifice ability to compress other kinds of data.”
That is not a meaningful limitation. There are general purpose universal compressors. It is part of the structure of reality that sequences generated by short programs are more commonly observed. That’s part of the point of using a compressor—it is an automated way of applying Occam’s razor.
There are frequently useful general purpose compressors that work by anticipating the most common regularities in the set of files typically generated by humans. But they do not, and cannot, iterate through all the short programs that could have generated the data—it’s too time-consuming.
The point was that general purpose compression is possible. Yes, you sacrifice the ability to compress other kinds of data—but those other kinds of data are highly incompressible and close to random—not the kind of data which most intelligent agents are interested in finding patterns in in the first place.
No, they look random and incompressible because effective compression algorithms optimized for this universe can’t compress them. But algorithms optimized for other computable universes may regard them as normal and have a good way to compress them.
Which kinds of data (from computable processes) are likely to be observed in this universe? Ay, there’s the rub.
Re: “they look random and incompressible because effective compression algorithms optimized for this universe can’t compress them”
Compressing sequences from this universe is good enough for me.
Re: “Which kinds of data (from computable processes) are likely to be observed in this universe? Ay, there’s the rub.”
Not really—there are well-known results about that—see:
http://en.wikipedia.org/wiki/Occam’s_razor
http://www.wisegeek.com/what-is-solomonoff-induction.htm
Except that the problem you were attacking at the beginning of this thread was general intelligence, which you claimed to be solvable just by good enough compression, but that requires knowing which parts of the search space in this universe are unlikely, which you haven’t shown how to algorithmize.
Yes, but as I keep trying to say, those results are far from enough to get something workable, and it’s not the methodology behind general compression programs.
Arithmetic compression, Huffman compression, Lempel-Ziv compression, etc are all excellent at compressing sequences produced by small programs. Things like:
1010101010101010 110110110110110110 1011011101111011111
...etc.
Those compressors (crudely) implement a computable approximation of Solomonoff induction without iterating through programs that generate the output. How they work is not very relevant here—the point is that they act as general-purpose compressors—and compress a great range of real world data types.
The complaint that we don’t know what types of data are in the universe is just not applicable—we do, in fact, know a considerable amount about that—and that is why we can build general purpose compressors.