Thanks so much for writing this, I think it’s a much needed—perhaps even a bit late contribution connecting static views of GPT-based LLMs to dynamical systems and predictive processing. I do research on empirical agency and it’s still surprises me how little the AI-safety community touches on this central part of agency—namely that you can’t have agents without this closed loop.
I’ve been speculating a bit (mostly to myself) about the possibility that “simulators” are already a type of organism—given that appear to do active inference—which is the main driving force for nervous system evolution. Simulators seem to live in this inter-dimensional paradigm where (i) on one hand during training they behave like (sensory-systems) agents because they learn to predict outcomes and “experience” the effect of their prediction; but (ii) during inference/prediction they generally do not receive feedback. As you point out, all of this speculation may be moot as many are moving pretty fast towards embedding simulators and giving them memory etc.
What is your opinion on this idea of “loosening up” our definition of agents? I spoke to Max Tegmark a few weeks ago and my position is that we might be thinking of organisms from a time-chauvinist position—where we require the loop to be closed in a fast fashion (e.g. 1sec for most biological organisms).
I do research on empirical agency and it’s still surprises me how little the AI-safety community touches on this central part of agency—namely that you can’t have agents without this closed loop.
In my view it’s one of the results of AI safety community being small and sort of bad in absorbing knowledge from elsewhere—my guess is this is in part a quirk due to founders effects, and also downstream of incentive structure on platforms like LessWrong.
But please do share this stuff.
I’ve been speculating a bit (mostly to myself) about the possibility that “simulators” are already a type of organism
...
What is your opinion on this idea of “loosening up” our definition of agents? I spoke to Max Tegmark a few weeks ago and my position is that we might be thinking of organisms from a time-chauvinist position—where we require the loop to be closed in a fast fashion (e.g. 1sec for most biological organisms).
I think we don’t have exact analogues of LLMs in existing systems, so there is a question where it’s better to extend the boundaries of some concepts, where to create new concepts.
I agree we are much more likely to use ‘intentional stance’ toward processes which are running on somewhat comparable time scales.
Thanks so much for writing this, I think it’s a much needed—perhaps even a bit late contribution connecting static views of GPT-based LLMs to dynamical systems and predictive processing. I do research on empirical agency and it’s still surprises me how little the AI-safety community touches on this central part of agency—namely that you can’t have agents without this closed loop.
I’ve been speculating a bit (mostly to myself) about the possibility that “simulators” are already a type of organism—given that appear to do active inference—which is the main driving force for nervous system evolution. Simulators seem to live in this inter-dimensional paradigm where (i) on one hand during training they behave like (sensory-systems) agents because they learn to predict outcomes and “experience” the effect of their prediction; but (ii) during inference/prediction they generally do not receive feedback. As you point out, all of this speculation may be moot as many are moving pretty fast towards embedding simulators and giving them memory etc.
What is your opinion on this idea of “loosening up” our definition of agents? I spoke to Max Tegmark a few weeks ago and my position is that we might be thinking of organisms from a time-chauvinist position—where we require the loop to be closed in a fast fashion (e.g. 1sec for most biological organisms).
Thanks for the comment.
In my view it’s one of the results of AI safety community being small and sort of bad in absorbing knowledge from elsewhere—my guess is this is in part a quirk due to founders effects, and also downstream of incentive structure on platforms like LessWrong.
But please do share this stuff.
I think we don’t have exact analogues of LLMs in existing systems, so there is a question where it’s better to extend the boundaries of some concepts, where to create new concepts.
I agree we are much more likely to use ‘intentional stance’ toward processes which are running on somewhat comparable time scales.