Rambling on the evolutionary development of feedback connections and efficiency in biological systems, and potential AI implications
My girlfriend and I were talking earlier about the fact that feedback connections in biological minds may have originally evolved as a way to deal with the fact that a human brain cannot have a thousand layers, there is just no space or energy for it, so you need to reroute information through the same neurons multiple times for simple lack of more neurons. And this is so interesting because feedback connections and loops seem to be crucial to the development of sentience. Maybe we got sentience as a byproduct of biology being pushed to optimise for efficiency, which then left us with sentience, and that turning out to be useful for intelligence, and because all biological life lives under these constraints, they all went on the same obvious efficiency route, which is why they all routed via sentience to intelligence.
But if artificial minds do not have these constraints and can simply massively scale up levels, maybe they can find a workaround for intelligence without sentience after all, because the efficiency pressure is so much lower?
At the conference my gf was at, one of the people talking about efficiency was saying how he is bloody sure that we are miles away from making our AI models properly efficient, and he is so sure because there are working models that are so much more efficient that it is ridiculous—and then threw a pic of human brain tissue on the wall. Brains are bizarrely cheap to run for the intelligence output they give. Heck, I can do meta reflections on my brain functions right now, and all that runs on… a couple biscuits.
The idea of sentience, consciousness, suffering, all the horror and pain there ever was, every beautiful and wondrous feeling, evolving not because they are the only path to intelligence, but because they are the cheapest biological path to intelligence, basically as a cost cutting measure to intelligence...
And that now also has me thinking that even if machines can route around it, and still get intelligence, that if it makes them more cost-efficient, we’ll still see them pushed to sentience.
One interesting thing corresponding to this is that most of our best models do have feedback loops using the same ‘neurons’ multiple times, feeding some or all of the output back into the input. The first token out of LLMs comes from a single pass through the model, but later passes can ‘see’ the previous output tokens, and the activations from those pass through the same ‘neurons’. Similar principles apply in diffusion models etc.
They are almost certainly very much larger than required for their capabilities though, which suggests that there might be quite a large capability overhang.
I’m aware that current artificial neural nets very much do have recurrent processing, and very interested in getting a proper understanding of how exactly feedback loops in artificial vs. natural neural nets differ in their implementation and function, because I suspect actually understanding that would be one way to get actual progress on whether sentient AI is a genuine risk or not. Like, a single feedback connection likely can’t do that much, and the different ways they can be implemented might matter. At which point exactly does the qualitative character of the system change with a massive improvement in performance? In my example above, I assumed that this was first just done to improve efficiency in getting responses already available, but that in the long run, it enabled minds to develop completely novel responses. I wonder if there is a similar transition in artificial systems.
Note that weight sharing (which is what I call reusing a neuron) also helps with statistical efficiency. That is, it takes less data to fit the weight to a certain accuracy.
Rambling on the evolutionary development of feedback connections and efficiency in biological systems, and potential AI implications
My girlfriend and I were talking earlier about the fact that feedback connections in biological minds may have originally evolved as a way to deal with the fact that a human brain cannot have a thousand layers, there is just no space or energy for it, so you need to reroute information through the same neurons multiple times for simple lack of more neurons. And this is so interesting because feedback connections and loops seem to be crucial to the development of sentience. Maybe we got sentience as a byproduct of biology being pushed to optimise for efficiency, which then left us with sentience, and that turning out to be useful for intelligence, and because all biological life lives under these constraints, they all went on the same obvious efficiency route, which is why they all routed via sentience to intelligence.
But if artificial minds do not have these constraints and can simply massively scale up levels, maybe they can find a workaround for intelligence without sentience after all, because the efficiency pressure is so much lower?
At the conference my gf was at, one of the people talking about efficiency was saying how he is bloody sure that we are miles away from making our AI models properly efficient, and he is so sure because there are working models that are so much more efficient that it is ridiculous—and then threw a pic of human brain tissue on the wall. Brains are bizarrely cheap to run for the intelligence output they give. Heck, I can do meta reflections on my brain functions right now, and all that runs on… a couple biscuits.
The idea of sentience, consciousness, suffering, all the horror and pain there ever was, every beautiful and wondrous feeling, evolving not because they are the only path to intelligence, but because they are the cheapest biological path to intelligence, basically as a cost cutting measure to intelligence...
And that now also has me thinking that even if machines can route around it, and still get intelligence, that if it makes them more cost-efficient, we’ll still see them pushed to sentience.
Interesting idea!
One interesting thing corresponding to this is that most of our best models do have feedback loops using the same ‘neurons’ multiple times, feeding some or all of the output back into the input. The first token out of LLMs comes from a single pass through the model, but later passes can ‘see’ the previous output tokens, and the activations from those pass through the same ‘neurons’. Similar principles apply in diffusion models etc.
They are almost certainly very much larger than required for their capabilities though, which suggests that there might be quite a large capability overhang.
I’m aware that current artificial neural nets very much do have recurrent processing, and very interested in getting a proper understanding of how exactly feedback loops in artificial vs. natural neural nets differ in their implementation and function, because I suspect actually understanding that would be one way to get actual progress on whether sentient AI is a genuine risk or not. Like, a single feedback connection likely can’t do that much, and the different ways they can be implemented might matter. At which point exactly does the qualitative character of the system change with a massive improvement in performance? In my example above, I assumed that this was first just done to improve efficiency in getting responses already available, but that in the long run, it enabled minds to develop completely novel responses. I wonder if there is a similar transition in artificial systems.
Note that weight sharing (which is what I call reusing a neuron) also helps with statistical efficiency. That is, it takes less data to fit the weight to a certain accuracy.