However, this also feels like a feature of the “valley of confused abstractions”. Humans didn’t evolve based on individual snapshots of reality, we evolved with moving pictures as input data. If a human in the ancestral environment, when faced with a tiger leaping out of the bushes, decided to analyse its fur patterns rather than register the large orange shape moving straight for them, it’s unlikely they would have returned from their first hunting trip. When we see a 2D image, we naturally “fill in the gaps”, and create a 3D model of it in our heads, but current image classifiers don’t do this, they just take a snapshot. For reasons of architectural bias, they are more likely to identify the abstraction of texture than the arguably far more natural abstraction of shape. Accordingly, we might expect video-based models to be more likely to recognise shapes than image-based models, and hence be more robust to this failure mode.
If this were true, then you’d expect NeRF’s and SORA to be a lot less biased towards textures, right?
If this were true, then you’d expect NeRF’s and SORA to be a lot less biased towards textures, right?