I think there’s a mistake which is being repeated in a few comments both here and on D𝜋′s post which needs emphasizing. Below is my understanding:
D𝜋 is attempting to create a general intelligence architecture. He is using image classification as a test for this general intelligence but his architecture is not optimized specifically for image identification.
Most attempts on MNIST use what we know about images (especially the importance of location of pixels) and design an architecture based on those facts. Convolutions are an especially obvious example of this. They are very effective at identifying images but the fact that we are inserting some of our knowledge of images into the algorithm precludes it from being a general intelligence methodology (without a lot of modification at least).
The point of using PI-MNIST (where locations of pixels in the dataset are randomized) is that we can’t use any of our own understanding of images to help with our model so a model which is good at PI-MNIST is proving a more general intelligence than a model which is good at MNIST.
That is why D𝜋 keeps on emphasizing that this is PI-MNIST.
I hope your explanation will be better understood than mine. Thank you.
It ‘so happens’ that MNIST (but not PI) can also be used for basic geometry. That is why I selected it for my exploration (easy switch between the two modes).
I think if one wants to test general intelligence, one should throw the algorithm at some problem that requires general intelligence. E.g. if it could reach SOTA on text prediction, that’d be impressive. But I think it would very badly fail at even approaching it, and I don’t see any obvious way to improve it.
I suppose it depends how general one is aiming to be. If by general intelligence we mean “able to do what a human can do” then no, at this point the method isn’t up to that standard.
If instead we mean “able to achieve SOTA on a difficult problem which it wasn’t specifically designed to deal with” then PI-MNIST seems like a reasonable starting point.
Also, from a practical standpoint PI-MNIST seems reasonable for a personal research project.
I do think D𝜋′s original post felt like it was overstating it’s case. From a later comment it seems like they more see it as a starting point to add more steps onto to achieve a more general intelligence (i.e. not just a scaling up of the same thing). So instead of paradigms which are MLP + others or DBM + others we would have S(O)NN + others.
I think there’s a mistake which is being repeated in a few comments both here and on D𝜋′s post which needs emphasizing. Below is my understanding:
D𝜋 is attempting to create a general intelligence architecture. He is using image classification as a test for this general intelligence but his architecture is not optimized specifically for image identification.
Most attempts on MNIST use what we know about images (especially the importance of location of pixels) and design an architecture based on those facts. Convolutions are an especially obvious example of this. They are very effective at identifying images but the fact that we are inserting some of our knowledge of images into the algorithm precludes it from being a general intelligence methodology (without a lot of modification at least).
The point of using PI-MNIST (where locations of pixels in the dataset are randomized) is that we can’t use any of our own understanding of images to help with our model so a model which is good at PI-MNIST is proving a more general intelligence than a model which is good at MNIST.
That is why D𝜋 keeps on emphasizing that this is PI-MNIST.
Spot on.
I hope your explanation will be better understood than mine. Thank you.
It ‘so happens’ that MNIST (but not PI) can also be used for basic geometry. That is why I selected it for my exploration (easy switch between the two modes).
I think if one wants to test general intelligence, one should throw the algorithm at some problem that requires general intelligence. E.g. if it could reach SOTA on text prediction, that’d be impressive. But I think it would very badly fail at even approaching it, and I don’t see any obvious way to improve it.
I suppose it depends how general one is aiming to be. If by general intelligence we mean “able to do what a human can do” then no, at this point the method isn’t up to that standard.
If instead we mean “able to achieve SOTA on a difficult problem which it wasn’t specifically designed to deal with” then PI-MNIST seems like a reasonable starting point.
Also, from a practical standpoint PI-MNIST seems reasonable for a personal research project.
I do think D𝜋′s original post felt like it was overstating it’s case. From a later comment it seems like they more see it as a starting point to add more steps onto to achieve a more general intelligence (i.e. not just a scaling up of the same thing). So instead of paradigms which are MLP + others or DBM + others we would have S(O)NN + others.