Subagents and impact measures: summary tables

[Stuart_Armstrong]

These tables will summarise the results of this whole sequence, checking whether subagents can neutralise the impact penalty.

First of all, given a subagent, here are the results for various impact penalties and baselines, and various “value difference summary functions” $f$:

$$\begin{array}{cccc}\text{Impact Penalty}& & \begin{array}{c}\text{Penalty neutralised}\\ \text{- sw inaction?}\end{array}& \begin{array}{c}\text{Penalty neutralised}\\ \text{- inaction?}\end{array}\\ \text{Non-indexical}& & \text{Yes}& \text{No}\\ \text{Indexical,}f\text{decreasing}& & \text{Yes}& \text{No}\\ \text{Indexical,}f\text{increasing}& & \text{Yes}& \text{Mostly}\end{array}$$

Another way of phrasing ”$f$ decreasing”: it penalises too little power, not too much. Conversely, ”$f$ increasing” penalises too much power, not too little. Thus, unfortunately:

• Subagents do allow an agent to get stronger than the indexical impact penalty would allow.

• Subagents don’t allow an agent to get weaker than the indexical impact penalty would allow.

Examples

This table presents, for three specific examples, whether they could actually build a subagent, and whether that would neutralise their impact penalty in practice (in the inaction baseline):

$$\begin{array}{cccc}\text{System}& & SA\text{possible?}& \text{Penalty neutralised?}\\ \text{20BQ}& & \text{Yes}& \text{No}\\ \text{RR}& & \text{Yes}& \text{No}\\ \text{AU}& & \text{Probably}& \text{Mostly}\end{array}$$

Here, 20BQ is twenty billion questions, RR is relative reachability, and AU is attainable utility.

Now, whether the RR or AU penalties are undermined technically depends on $f$, not on what measure is being used for value. However, I feel that the results undermine the spirit of AU much more than the spirit of RR. AU attempted to control an agent by limiting its power; this effect is mainly neutralised. RR attempted to control the side-effects of an agent by ensuring it had enough power to reach a lot of states; this effect is not neutralised by a subagent.