A program that could compress any type of non-random data could be improved significantly just by focusing on thing that are easy to predict.
Not by very much usually, if there is lots of data. Powerful general purpose compressors would actually work well.
Anyway, that is the idea—to build a general purpose Occamian system—not one with lots of preconceptions. Non-Occamian preconceptions don’t buy that much and are not really needed—if you are sufficiently smart and have a little while to look at the world and see what it is like.
A program that could compress any type of non-random data could be improved significantly just by focusing on thing that are easy to predict.
Not by very much usually, if there is lots of data. Powerful general purpose compressors would actually work well.
Huffman encoding also works very well in many cases. Obviously you cannot compress any compressible data without GAI, but there are no programs in existence that can compress any compressible data and things like Huffman encoding do make numerical progress in compression, but they do not represent any real conceptual progress. Do you know of any compression programs that make conceptual progress toward GAI? If not, then why do you think that focusing on the compression aspect is likely to provide such progress?
I was just trying to make the point that compression progress is possible without AI progress.
If you have a compressor that isn’t part of a machine intelligence, or is part of a non-state of the art machine intelligence, then that is likely to be true.
However, if your compressor is in a state of the art machine intelligence—and it is the primary tool being used to predict the consequences of its actions, then
compression progress (smaller or faster), would translate into increased intelligence. It would help the machine to better predict the consequences of its possible actions, and/or to act more quickly.
However, if your compressor is in a state of the art machine intelligence—and it is the primary tool being used to predict the consequences of its actions, then compression progress (smaller or faster), would translate into increased intelligence. It would help the machine to better predict the consequences of its possible actions, and/or to act more quickly.
What techniques does he use that teach us something that could be useful for GAI?
Alas, delving in is beyond the scope of this comment.
Your arguments seem to be more inside-view than mine, so I will update my estimates in favour of your point.
I got something from you too. One of the problems with a compression-based approach to machine intelligence is that so far, it hasn’t been very popular. There just aren’t very many people working on it.
Compression is a tough traditional software engineering problem. It seems relatively unglamourous—and there are some barriers to entry, in the form of a big mountain of existing work. Building on that work might not be the most direct way towards the goal—but unless you do that, you can’t easily make competitive products in the field.
Sequence prediction (via stream compression) still seems like the number 1 driving problem to me—and a likely path towards the goal—but some of the above points do seem to count against it.
It is not clear to me that what we are measuring is progress.
Compression ratio, on general Occamian data defined w.r.t a small reference machine. If in doubt, refer to Shane Legg: http://www.vetta.org/publications/
Not by very much usually, if there is lots of data. Powerful general purpose compressors would actually work well.
Anyway, that is the idea—to build a general purpose Occamian system—not one with lots of preconceptions. Non-Occamian preconceptions don’t buy that much and are not really needed—if you are sufficiently smart and have a little while to look at the world and see what it is like.
Huffman encoding also works very well in many cases. Obviously you cannot compress any compressible data without GAI, but there are no programs in existence that can compress any compressible data and things like Huffman encoding do make numerical progress in compression, but they do not represent any real conceptual progress. Do you know of any compression programs that make conceptual progress toward GAI? If not, then why do you think that focusing on the compression aspect is likely to provide such progress?
Huffman coding dates back to 1952 - it has been replaced by othehr schemes in size-sensitive applications.
Alexander Ratushnyak seems to be making good progress—see: http://prize.hutter1.net/
I realize that Huffman coding is outdated, I was just trying to make the point that compression progress is possible without AI progress.
Do you have Alexander Ratushnyak’s source code? What techniques does he use that teach us something that could be useful for GAI?
If you have a compressor that isn’t part of a machine intelligence, or is part of a non-state of the art machine intelligence, then that is likely to be true.
However, if your compressor is in a state of the art machine intelligence—and it is the primary tool being used to predict the consequences of its actions, then compression progress (smaller or faster), would translate into increased intelligence. It would help the machine to better predict the consequences of its possible actions, and/or to act more quickly.
That is available here.
Alas, delving in is beyond the scope of this comment.
Your arguments seem to be more inside-view than mine, so I will update my estimates in favour of your point.
I got something from you too. One of the problems with a compression-based approach to machine intelligence is that so far, it hasn’t been very popular. There just aren’t very many people working on it.
Compression is a tough traditional software engineering problem. It seems relatively unglamourous—and there are some barriers to entry, in the form of a big mountain of existing work. Building on that work might not be the most direct way towards the goal—but unless you do that, you can’t easily make competitive products in the field.
Sequence prediction (via stream compression) still seems like the number 1 driving problem to me—and a likely path towards the goal—but some of the above points do seem to count against it.