Downvoted for sheer repetitive drudgery, without genuine motivation for the reader. How many chapters before you show us a real-world instance of CRM achieving progress in research?
When Eliezer had a sequence to write, each post was (1) interesting to read on its own, (2) connected to some real event/interesting story/experimental data, and (3) clearly adding something new to the conversation. You would do well to work on those aspects of your writing.
For my own clarification, did you think the idea of using CRM to evaluate theories of linguistics was uninteresting, or did you just not believe me that it would work?
The latter. Until I see a real-world case where CRM has been very effective compared to other methods, I’m not going to give much credit to claims that it will achieve greatness in this, that and the other field.
And in particular, I find it extremely unlikely that current major theories of linguistics could in practice be coded into compressors, in a way that satisfies their proponents.
This isn’t precisely what Daniel_Burfoot was talking about but its a related idea based on “sparse coding” and it has recently obtained good results in classification:
Here the “theories” are hierarchical dictionaries (so a discrete hierarchy index set plus a set of vectors) which perform a compression (by creating reconstructions of the data). Although they weren’t developed with this in mind, support vector machines also do this as well, since one finds a small number of “support vectors” that essentially allow you to compress the information about decision boundaries in classification problems (support vector machines are one of the very few things from machine learning that have had significant and successful impacts elsewhere since neural networks).
The hierarchical dictionaries learned do contain a “theory” of the visual world in a sense, although an important idea is that they do so in a way that is sensitive to the application at hand. There is much left out by Daniel_Burfoot about how people actually go about implementing this line of thought.
For me, along with everything orthonormal and I said before, the problem here is that your application of CRM to linguistics is just:
“CRM is a good epistemology (which the entire reader base already agrees with). Here’s the obvious things you would do if you applied it to linguistics. Don’t worry, something new is coming, just wait a few more articles.”
It’s a new idea to use the CRM to approach computer vision. Nobody in computer vision thinks about large scale lossless compression of natural images. All work in CV is predicated on the idea that good methods can be obtained through pure theorizing and algorithm design. Consider for example this famous CV paper. Note the complex mathematics and algorithm design, and then note that the empirical verification consists of the application of the method to about four images (the reader is supposed to verify that the segmentation result agrees with intuition). Or check out this rival technique which actually uses MDL but doesn’t report the actual compression rates, only the segmentation results.
If that was your point, you could have resolved this in one article whose thesis is just, “Hey, like you already know, MML is a superior method of rationality compared to traditional science. [Here’s a review of MML.] Behold, the best work in computer vision ignores it! Look how much better you’d do, just by reading this site!”
You didn’t need to go into all the fluff about AI’s advances and failures, what’s wrong with science, blah blah blah.
(Not to say this doesn’t benefit you; you seem to get net positive karma, with the x10 modifier, each time you draw this out to yet another post, despite not presenting anything new.)
Of course, I thought your original motivation for this series was to explain what super-powerful epistemology it is babies must be using to be able to go from a near-blank slate[1] to solving AI-complete problems on limited empirical data, and how we can replicate that. And you haven’t even touched that one.
[1] which I, and the best scientific research, dispute is an accurate characterization of babies
Downvoted for sheer repetitive drudgery, without genuine motivation for the reader. How many chapters before you show us a real-world instance of CRM achieving progress in research?
When Eliezer had a sequence to write, each post was (1) interesting to read on its own, (2) connected to some real event/interesting story/experimental data, and (3) clearly adding something new to the conversation. You would do well to work on those aspects of your writing.
For my own clarification, did you think the idea of using CRM to evaluate theories of linguistics was uninteresting, or did you just not believe me that it would work?
The latter. Until I see a real-world case where CRM has been very effective compared to other methods, I’m not going to give much credit to claims that it will achieve greatness in this, that and the other field.
And in particular, I find it extremely unlikely that current major theories of linguistics could in practice be coded into compressors, in a way that satisfies their proponents.
This isn’t precisely what Daniel_Burfoot was talking about but its a related idea based on “sparse coding” and it has recently obtained good results in classification:
http://www.di.ens.fr/~fbach/icml2010a.pdf
Here the “theories” are hierarchical dictionaries (so a discrete hierarchy index set plus a set of vectors) which perform a compression (by creating reconstructions of the data). Although they weren’t developed with this in mind, support vector machines also do this as well, since one finds a small number of “support vectors” that essentially allow you to compress the information about decision boundaries in classification problems (support vector machines are one of the very few things from machine learning that have had significant and successful impacts elsewhere since neural networks).
The hierarchical dictionaries learned do contain a “theory” of the visual world in a sense, although an important idea is that they do so in a way that is sensitive to the application at hand. There is much left out by Daniel_Burfoot about how people actually go about implementing this line of thought.
For me, along with everything orthonormal and I said before, the problem here is that your application of CRM to linguistics is just:
“CRM is a good epistemology (which the entire reader base already agrees with). Here’s the obvious things you would do if you applied it to linguistics. Don’t worry, something new is coming, just wait a few more articles.”
It’s a new idea to use the CRM to approach computer vision. Nobody in computer vision thinks about large scale lossless compression of natural images. All work in CV is predicated on the idea that good methods can be obtained through pure theorizing and algorithm design. Consider for example this famous CV paper. Note the complex mathematics and algorithm design, and then note that the empirical verification consists of the application of the method to about four images (the reader is supposed to verify that the segmentation result agrees with intuition). Or check out this rival technique which actually uses MDL but doesn’t report the actual compression rates, only the segmentation results.
If that was your point, you could have resolved this in one article whose thesis is just, “Hey, like you already know, MML is a superior method of rationality compared to traditional science. [Here’s a review of MML.] Behold, the best work in computer vision ignores it! Look how much better you’d do, just by reading this site!”
You didn’t need to go into all the fluff about AI’s advances and failures, what’s wrong with science, blah blah blah.
(Not to say this doesn’t benefit you; you seem to get net positive karma, with the x10 modifier, each time you draw this out to yet another post, despite not presenting anything new.)
Of course, I thought your original motivation for this series was to explain what super-powerful epistemology it is babies must be using to be able to go from a near-blank slate[1] to solving AI-complete problems on limited empirical data, and how we can replicate that. And you haven’t even touched that one.
[1] which I, and the best scientific research, dispute is an accurate characterization of babies
As almost everything else, much less interesting, wouldn’t be said 100 times?
I like it. Probably be cause I mostly agree.