The more agentic a system is the more it is likely to adopt convergent instrumental goals, yes.
Why agents are powerful explores why agentic mesa optimizers might arise accidentally during training. In particular, agents are an efficient way to solve many challenges, so the mesa optimizer being resource constrained would lean in the direction of more agency under some circumstances.
The more agentic a system is the more it is likely to adopt convergent instrumental goals, yes.
Why agents are powerful explores why agentic mesa optimizers might arise accidentally during training. In particular, agents are an efficient way to solve many challenges, so the mesa optimizer being resource constrained would lean in the direction of more agency under some circumstances.