One interesting thing corresponding to this is that most of our best models do have feedback loops using the same ‘neurons’ multiple times, feeding some or all of the output back into the input. The first token out of LLMs comes from a single pass through the model, but later passes can ‘see’ the previous output tokens, and the activations from those pass through the same ‘neurons’. Similar principles apply in diffusion models etc.
They are almost certainly very much larger than required for their capabilities though, which suggests that there might be quite a large capability overhang.
I’m aware that current artificial neural nets very much do have recurrent processing, and very interested in getting a proper understanding of how exactly feedback loops in artificial vs. natural neural nets differ in their implementation and function, because I suspect actually understanding that would be one way to get actual progress on whether sentient AI is a genuine risk or not. Like, a single feedback connection likely can’t do that much, and the different ways they can be implemented might matter. At which point exactly does the qualitative character of the system change with a massive improvement in performance? In my example above, I assumed that this was first just done to improve efficiency in getting responses already available, but that in the long run, it enabled minds to develop completely novel responses. I wonder if there is a similar transition in artificial systems.
Note that weight sharing (which is what I call reusing a neuron) also helps with statistical efficiency. That is, it takes less data to fit the weight to a certain accuracy.
Interesting idea!
One interesting thing corresponding to this is that most of our best models do have feedback loops using the same ‘neurons’ multiple times, feeding some or all of the output back into the input. The first token out of LLMs comes from a single pass through the model, but later passes can ‘see’ the previous output tokens, and the activations from those pass through the same ‘neurons’. Similar principles apply in diffusion models etc.
They are almost certainly very much larger than required for their capabilities though, which suggests that there might be quite a large capability overhang.
I’m aware that current artificial neural nets very much do have recurrent processing, and very interested in getting a proper understanding of how exactly feedback loops in artificial vs. natural neural nets differ in their implementation and function, because I suspect actually understanding that would be one way to get actual progress on whether sentient AI is a genuine risk or not. Like, a single feedback connection likely can’t do that much, and the different ways they can be implemented might matter. At which point exactly does the qualitative character of the system change with a massive improvement in performance? In my example above, I assumed that this was first just done to improve efficiency in getting responses already available, but that in the long run, it enabled minds to develop completely novel responses. I wonder if there is a similar transition in artificial systems.
Note that weight sharing (which is what I call reusing a neuron) also helps with statistical efficiency. That is, it takes less data to fit the weight to a certain accuracy.