OK, thanks for linking that. You’re probably right in the specific example of MNIST. I’m less convinced about more complicated tasks—it seems like each individual task would require a lot of engineering effort.
One thing I didn’t see—is there research which looks at what happens if you give neural nets more of the input space as data? Things which are explicitly out-of-distribution, random noise, abstract shapes, or maybe other modes that you don’t particularly care about performance on, and label it all as “garbage” or whatever. Essentially, providing negative as well as positive examples, given that the input spaces are usually much larger than the intended distribution.
OK, thanks for linking that. You’re probably right in the specific example of MNIST. I’m less convinced about more complicated tasks—it seems like each individual task would require a lot of engineering effort.
One thing I didn’t see—is there research which looks at what happens if you give neural nets more of the input space as data? Things which are explicitly out-of-distribution, random noise, abstract shapes, or maybe other modes that you don’t particularly care about performance on, and label it all as “garbage” or whatever. Essentially, providing negative as well as positive examples, given that the input spaces are usually much larger than the intended distribution.