I know you’ve acknowledged Friston at the end, but I’m just commenting for other interested readers’ benefit that this is very close to Karl Friston’s active inference framework, which posits that all agents minimise the discrepancies (or prediction errors) between their internal representations of the world and their incoming sensory information through both action and perception.
James Fox
Karma: 295
AI Alignment Research Engineer Accelerator (ARENA): Call for applicants v4.0
Announcing the London Initiative for Safe AI (LISA)
Reward Hacking from a Causal Perspective
Incentives from a causal perspective
Agency from a causal perspective
Causality: A Brief Introduction
Introduction to Towards Causal Foundations of Safe AGI
Hi Vanessa, Thanks for your question! Sorry for taking a while to reply. The answer is yes if we allow for mixed policies (i.e., where an agent can correlate all of their decision rules for different decisions with a shared random bit), but no if we restrict agents to only be able to use behavioural policies (i.e., decision rules for each of an agent’s decisions are independent because they can’t access a shared random bit). This is analogous to the difference between mixed and behavioural strategies in extensive form games, where (in general) a subgame perfect equilibrium (SPE) is only guaranteed to exist in mixed strategies (and the game is finite etc by Nash’ theorem).
Note that If all agents in the MAIM have perfect recall (where they remember their previous decisions and the information that they knew at previous decisions), then there is guaranteed to exist a SPE in behavioural policies). In fact, Koller and Milch showed that only a weaker criterion of “sufficient recall” is needed (https://www.semanticscholar.org/paper/Ignorable-Information-in-Multi-Agent-Scenarios-Milch-Koller/5ea036bad72176389cf23545a881636deadc4946).
In a forthcoming journal paper, we expand significantly on the the theoretical underpinnings and advantages of MAIMs and so we will provide more results there.
Sorry for not seeing this. Hopefully, the first paragraph of the summary answers this question. We’re excited about running more ARENA iterations exactly because its track record has been pretty strong.