Then upon seeing a purple flower, the node np is strongly activated for the first time, and a whole series of weight changes flow across mAIry’s brain, making cp true.
Another way of describing what happens: performing the update may push the weights into a whole new part of model space. mAIry may have been able to predict what the update would be, roughly, but wouldn’t have a memory of having had their brain inhabit this part of the model space. For many long-term memory setups, it would probably make sense for an agent to remember this kind of update as qualitatively different from past experience.
Another way of describing what happens: performing the update may push the weights into a whole new part of model space. mAIry may have been able to predict what the update would be, roughly, but wouldn’t have a memory of having had their brain inhabit this part of the model space. For many long-term memory setups, it would probably make sense for an agent to remember this kind of update as qualitatively different from past experience.
I like that way of seeing it.