The trippy shuggorth title image was mysterious when it was originally posted, basically someone leaked an image a little before the inceptionism blog post.
A CNN is a reasonable model for fast feedforward vision. We can isolate this pathway for biological vision by using rapid serial presentation—basically flashing an image for 100ms or so.
So imagine if you just saw a flash of one of these images, for a brief moment, and then you had to quickly press a button for the image category—no time to think about it—it’s jeopardy style instant response.
There is no button for “noisy image”, there is no button for “wavy line image”, etc.
Now the fooling images are generated by an adversarial process. It’s like we have a copy of a particular mind in a VR sim, we flash it an image, see what button it presses. Based on the response, we then generate a new image and unwind time and repeat. We keep doing this until we get some wierd classification errors. It allows us to explore the decision space of the agent.
It is basically reverse engineering. It requires a copy of the agent’s code or at least access to a copy with the ability to do tons of queries, and it also probably depends on the agent being completely deterministic. I think that biological minds avoid this issue indirectly because they use stochastic sampling based on secure hardware/analog noise generators.
Stochastic models/ANNs could probably avoid this issue.
The trippy shuggorth title image was mysterious when it was originally posted, basically someone leaked an image a little before the inceptionism blog post.
A CNN is a reasonable model for fast feedforward vision. We can isolate this pathway for biological vision by using rapid serial presentation—basically flashing an image for 100ms or so.
So imagine if you just saw a flash of one of these images, for a brief moment, and then you had to quickly press a button for the image category—no time to think about it—it’s jeopardy style instant response.
There is no button for “noisy image”, there is no button for “wavy line image”, etc.
Now the fooling images are generated by an adversarial process. It’s like we have a copy of a particular mind in a VR sim, we flash it an image, see what button it presses. Based on the response, we then generate a new image and unwind time and repeat. We keep doing this until we get some wierd classification errors. It allows us to explore the decision space of the agent.
It is basically reverse engineering. It requires a copy of the agent’s code or at least access to a copy with the ability to do tons of queries, and it also probably depends on the agent being completely deterministic. I think that biological minds avoid this issue indirectly because they use stochastic sampling based on secure hardware/analog noise generators.
Stochastic models/ANNs could probably avoid this issue.