There’s an old Google tech talk by Geoff Hinton about restricted Boltzmann machines that was my introduction to a bunch of topics in machine learning, and it still strongly informs a lot of my thinking about cognition. The main example is about labeling image from the MNIST handwritten digit database. Then, starting around 23:20, he shows that his RBMs are also generative; he can fix the state of a high level unit in the network (that normally discriminated images of 2s), crank forward his inference algorithm, and get out images of handwritten numbers “2”. He says that the patterns of activation on the high level units are brain states, and the generated images are mental states, i.e. what the model is thinking about, because those are the sorts of inputs the environment would have to supply to the model for the model to be perceiving an image of a number two.
Is that perspective really obvious to everyone? That mental imagery is like counterfactual simulation with top down biased activations? And patterns of neural oscillations are like Gibbs sampling at work? And, at least in the case of single modality visual perception, you can point to a model and say what it’s thinking, by squeezing the perception out back through the feature detectors till it becomes sensory data once more?
I’d been thinking about writing this up for a while, but wanted to learn some more about how RBM inference and learning map onto bayesian networks or HMMs, whether we can generate more interesting data like photorealistic images or videos, what’s left to be done in machine vision, how adequate our models of the primate visual cortex are, et cetera. Now that Eliezer has posted Causal Reference, I think a post like that might be redundant, in regard to demystifying.
Have you seen Prof. Hinton’s neural networks class on Coursera, or equivalent?
If you can’t get access to the materials now (since I don’t think there’s a scheduled re-run of the class yet), I can make them available to you if you want them.
There’s an old Google tech talk by Geoff Hinton about restricted Boltzmann machines that was my introduction to a bunch of topics in machine learning, and it still strongly informs a lot of my thinking about cognition. The main example is about labeling image from the MNIST handwritten digit database. Then, starting around 23:20, he shows that his RBMs are also generative; he can fix the state of a high level unit in the network (that normally discriminated images of 2s), crank forward his inference algorithm, and get out images of handwritten numbers “2”. He says that the patterns of activation on the high level units are brain states, and the generated images are mental states, i.e. what the model is thinking about, because those are the sorts of inputs the environment would have to supply to the model for the model to be perceiving an image of a number two.
Is that perspective really obvious to everyone? That mental imagery is like counterfactual simulation with top down biased activations? And patterns of neural oscillations are like Gibbs sampling at work? And, at least in the case of single modality visual perception, you can point to a model and say what it’s thinking, by squeezing the perception out back through the feature detectors till it becomes sensory data once more?
I’d been thinking about writing this up for a while, but wanted to learn some more about how RBM inference and learning map onto bayesian networks or HMMs, whether we can generate more interesting data like photorealistic images or videos, what’s left to be done in machine vision, how adequate our models of the primate visual cortex are, et cetera. Now that Eliezer has posted Causal Reference, I think a post like that might be redundant, in regard to demystifying.
Have you seen Prof. Hinton’s neural networks class on Coursera, or equivalent?
If you can’t get access to the materials now (since I don’t think there’s a scheduled re-run of the class yet), I can make them available to you if you want them.