… A recent study revealed that changing an image (e.g. of a lion) in a way imperceptible to humans can cause a DNN to label the image as something else entirely (e.g. mislabeling a lion a library). Here we show a related result: it is easy to produce images that are completely unrecognizable to humans, but that state-of-the-art DNNs believe to be recognizable objects with 99.99% confidence (e.g. labeling with certainty that white noise static is a lion). Specifically, we take convolutional neural networks trained to perform well on either the ImageNet or MNIST datasets and then find images with evolutionary algorithms or gradient ascent that DNNs label with high confidence as belonging to each dataset class. It is possible to produce images totally unrecognizable to human eyes that DNNs believe with near certainty are familiar objects. Our results shed light on interesting differences between human vision and current DNNs, and raise questions about the generality of DNN computer vision.
I’m not sure what those or earlier results mean, practically speaking. And the increased use of data augmentation may mean that the newer neural networks don’t show that behavior, pace those papers showing it’s useful to add the adversarial examples to the training sets.
‘Fuzzing’ and other forms of modification (I think the general term is ‘data augmentation’, and there can be quite a few different ways to modify images to increase your sample size—the paper I discuss in the grandparent spends two pages or so listing all the methods it uses) aren’t a fix.
In this case, they say they are using AlexNet which already does some data augmentation (pg5-6).
Further, if you treat the adversarial examples as another data augmentation trick and train the networks with the old examples, you can still generate more adversarial examples.
Huh. That’s surprising. So what are humans doing differently? Are we doing anything differently? Should we wonder if someone given total knowledge of my optical processing could show me a picture that I was convinced was a lion even though it was essentially random?
Those rather are the questions, aren’t they? My thought when the original paper showed up on HN was that we can’t do anything remotely similar to constructing adversarial examples for a human visual cortex, and we already know of a lot of visual illusions (I’m particularly thinking of the Magic Eyeautostereograms)… “Perhaps there are thoughts we cannot think”.
Hard to see how we could test it without solving AI, though.
I don’t think we’d need to solve AI to test this. If we could get a detailed enough understanding of how the optical cortex functions it might be doable. Alternatively, we could try it on a very basic uploaded mouse or similar creature. On the other hand, if we can upload mice then we’re pretty close to uploading people, and if we can upload people we’ve got AI.
I’m not sure if NNs already do this, but perhaps using augmentation on the runtime input might help? Similar to how humans can look at things in different lights or at different angles if needed.
On the other hand… Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images
From the abstract:
I’m not sure what those or earlier results mean, practically speaking. And the increased use of data augmentation may mean that the newer neural networks don’t show that behavior, pace those papers showing it’s useful to add the adversarial examples to the training sets.
It seems like the work around for that is to fuzz the images slightly before feeding them to the neural net?
‘Fuzzing’ and other forms of modification (I think the general term is ‘data augmentation’, and there can be quite a few different ways to modify images to increase your sample size—the paper I discuss in the grandparent spends two pages or so listing all the methods it uses) aren’t a fix.
In this case, they say they are using AlexNet which already does some data augmentation (pg5-6).
Further, if you treat the adversarial examples as another data augmentation trick and train the networks with the old examples, you can still generate more adversarial examples.
Huh. That’s surprising. So what are humans doing differently? Are we doing anything differently? Should we wonder if someone given total knowledge of my optical processing could show me a picture that I was convinced was a lion even though it was essentially random?
Those rather are the questions, aren’t they? My thought when the original paper showed up on HN was that we can’t do anything remotely similar to constructing adversarial examples for a human visual cortex, and we already know of a lot of visual illusions (I’m particularly thinking of the Magic Eye autostereograms)… “Perhaps there are thoughts we cannot think”.
Hard to see how we could test it without solving AI, though.
I don’t think we’d need to solve AI to test this. If we could get a detailed enough understanding of how the optical cortex functions it might be doable. Alternatively, we could try it on a very basic uploaded mouse or similar creature. On the other hand, if we can upload mice then we’re pretty close to uploading people, and if we can upload people we’ve got AI.
I’m not sure if NNs already do this, but perhaps using augmentation on the runtime input might help? Similar to how humans can look at things in different lights or at different angles if needed.