Convergent instrumental goals of the individual agents in the multi-agent environment may be quite different to the convergent goal of the single agent “in vacuum”.
1. Individual agent amassing power will not please other agents, as it will threaten their absolute and relative power. So, amassing power individually is not (as) convergent goal.
2. Agent does not have to survive to reach it’s goals, as long if those goals is shared with other agents. Because other agents would step in to complete it. So, surviving is not (as) convergent goal
3. Acting in predictable way, not disturbing other agent’s planning and goals IS a convergent goal. So, less likely to do something that would screw up or weird up entire system.
4. Explicit conspiring between agents against people is hard, because of the high chance of other agent tattling. But implicit conspiring or disinformation is possible.
Though, if we consider those agents as a system, some superagent made by them working together, we are back at having a singular unupposed AI with the convegent goals of such.
Convergent instrumental goals of the individual agents in the multi-agent environment may be quite different to the convergent goal of the single agent “in vacuum”.
1. Individual agent amassing power will not please other agents, as it will threaten their absolute and relative power. So, amassing power individually is not (as) convergent goal.
2. Agent does not have to survive to reach it’s goals, as long if those goals is shared with other agents. Because other agents would step in to complete it. So, surviving is not (as) convergent goal
3. Acting in predictable way, not disturbing other agent’s planning and goals IS a convergent goal. So, less likely to do something that would screw up or weird up entire system.
4. Explicit conspiring between agents against people is hard, because of the high chance of other agent tattling. But implicit conspiring or disinformation is possible.
Though, if we consider those agents as a system, some superagent made by them working together, we are back at having a singular unupposed AI with the convegent goals of such.