A common trope is that brains are trained on prediction. Well, technically, I claim it would be more accurate to say that they’re trained on postdiction. Like, let’s say I open a package, expecting to see a book, but it’s actually a screwdriver. I’m surprised, and I immediately update my world-model to say “the box has a screwdriver in it”.
I would argue that that the book-expectation is a prediction, and the surprise you experience is a result of low mutual information between your retinal activation patterns and the book-expectation in your head. That surprise (prediction error) is the learning signal that propagates up to your more abstract world-model, which updates into a state consistent with “the box has a screwdriver in it”.
During this process, there was a moment when I was just beginning to parse the incoming image of the newly-opened box, expecting to see a book inside. A fraction of a second later, my brain recognizes that my expectation was wrong, and that’s the “surprise”. So in other words, my brain had an active expectation about something that had already happened—about photons that had by then already arrived at my retina—and that expectation was incorrect, and that’s what spurred me to update my world-model. Postdiction, not prediction.
Right, but the part of your brain that had that high-level model of “there is a book in the box” had at that time not received contradictory input from the lower-level edge detection / feature extraction areas. The abstract world-model does not directly predict retinal activations, it predicts the activations of lower-level sensory processing areas, which in turn predict the activations of the retina, cochlea, etc. There is latency in this system, so the signal takes a bit of time to filter up from the retinas to lower-level visual areas to your world-model. I don’t think ‘post-diction’ makes sense in this context, as each brain region is predicting the activations of the one below it, and updates its state when those predictions are wrong.
(Also, I think Easy Win #3 is a really good point for Predict-O-Matic-esque sytems)
I think you’re interpreting “prediction” and “postdiction” differently than me.
Like, let’s say GPT-3 is being trained to guess the next word of a text. You mask (hide) the next word, have GPT-3 guess it, and then compare the masked word to the guess and make an update.
I think you want to call the guess a “prediction” because from GPT-3′s perspective, the revelation of the masked data is something that hasn’t happened yet. But I want to call the guess a “postdiction” because the masked data is already “locked in” at the time that the guess is formed. The latter is relevant when we’re thinking about incentives to form self-fulfilling prophecies.
Incidentally, to be clear, people absolutely do make real predictions constantly. I’m just saying we don’t train on those predictions. I’m saying that by the time the model update occurs, the predictions have already been transmuted into postdictions, because the thing-that-was-predicted has now already been “locked in”.
Nope, that’s an accurate representation of my views. If “postdiction” means “the machine has no influence over its sensory input”, then yeah, that’s a really good idea.
There are 2 ways to reduce prediction error: change your predictions, or act upon the environment to make your predictions come true. I think the agency of an entity depends on how much of each they do. An entity with no agency would have no influence over its sensory inputs, instead opting to update beliefs in the face of prediction error. Taking agency from AIs is a good idea for safety.
Scott Alexander recently wrote about a similar quantity being ecoded in humans through the 5-HT1A / 5-HT2A receptor activation ratio: link
I would argue that that the book-expectation is a prediction, and the surprise you experience is a result of low mutual information between your retinal activation patterns and the book-expectation in your head. That surprise (prediction error) is the learning signal that propagates up to your more abstract world-model, which updates into a state consistent with “the box has a screwdriver in it”.
Right, but the part of your brain that had that high-level model of “there is a book in the box” had at that time not received contradictory input from the lower-level edge detection / feature extraction areas. The abstract world-model does not directly predict retinal activations, it predicts the activations of lower-level sensory processing areas, which in turn predict the activations of the retina, cochlea, etc. There is latency in this system, so the signal takes a bit of time to filter up from the retinas to lower-level visual areas to your world-model. I don’t think ‘post-diction’ makes sense in this context, as each brain region is predicting the activations of the one below it, and updates its state when those predictions are wrong.
(Also, I think Easy Win #3 is a really good point for Predict-O-Matic-esque sytems)
I think you’re interpreting “prediction” and “postdiction” differently than me.
Like, let’s say GPT-3 is being trained to guess the next word of a text. You mask (hide) the next word, have GPT-3 guess it, and then compare the masked word to the guess and make an update.
I think you want to call the guess a “prediction” because from GPT-3′s perspective, the revelation of the masked data is something that hasn’t happened yet. But I want to call the guess a “postdiction” because the masked data is already “locked in” at the time that the guess is formed. The latter is relevant when we’re thinking about incentives to form self-fulfilling prophecies.
Incidentally, to be clear, people absolutely do make real predictions constantly. I’m just saying we don’t train on those predictions. I’m saying that by the time the model update occurs, the predictions have already been transmuted into postdictions, because the thing-that-was-predicted has now already been “locked in”.
(Sorry if I’m misunderstanding.)
Nope, that’s an accurate representation of my views. If “postdiction” means “the machine has no influence over its sensory input”, then yeah, that’s a really good idea.
There are 2 ways to reduce prediction error: change your predictions, or act upon the environment to make your predictions come true. I think the agency of an entity depends on how much of each they do. An entity with no agency would have no influence over its sensory inputs, instead opting to update beliefs in the face of prediction error. Taking agency from AIs is a good idea for safety.
Scott Alexander recently wrote about a similar quantity being ecoded in humans through the 5-HT1A / 5-HT2A receptor activation ratio: link