I think this question sort of misses what matters.
There’s all sorts of computations which (probably) aren’t very interpretable; SHA-256 is a solid example. But it’s an empirical fact that our physical world has a lot more interpretable structure in it than SHA-256 computations. We have things like trees or cars, large-scale abstract structures which repeat over and over again, and display similar predictable behavior across instances despite different small-scale configurations.
Trained neural networks are not basically-random computations (like SHA-256); they’re trained on the real world. We know that the real world has a lot of interpretable structure, so it’s feasible that a network trained on the real world will reflect that structure. That’s what Olah et al’s research is about—backing out the structure of the real world from a network trained on the real world.
It’s the coupling of the (trained) network to the real world which plays the central role. Something like Conway’s game of life doesn’t have any coupling to the real world, so it’s not really analogous.
I think also Conway’s game of life has a large bestiary of ‘stable patterns’ that you could figure out and then dramatically increase your ability to predict things.
As is demonstrated by the Hashlife algorithm, that exploits the redundancies for a massive speedup. That’s not possible for things like SHA-256 (by design)!
I think this question sort of misses what matters.
There’s all sorts of computations which (probably) aren’t very interpretable; SHA-256 is a solid example. But it’s an empirical fact that our physical world has a lot more interpretable structure in it than SHA-256 computations. We have things like trees or cars, large-scale abstract structures which repeat over and over again, and display similar predictable behavior across instances despite different small-scale configurations.
Trained neural networks are not basically-random computations (like SHA-256); they’re trained on the real world. We know that the real world has a lot of interpretable structure, so it’s feasible that a network trained on the real world will reflect that structure. That’s what Olah et al’s research is about—backing out the structure of the real world from a network trained on the real world.
It’s the coupling of the (trained) network to the real world which plays the central role. Something like Conway’s game of life doesn’t have any coupling to the real world, so it’s not really analogous.
I think also Conway’s game of life has a large bestiary of ‘stable patterns’ that you could figure out and then dramatically increase your ability to predict things.
As is demonstrated by the Hashlife algorithm, that exploits the redundancies for a massive speedup. That’s not possible for things like SHA-256 (by design)!