I notice being confused about the relationship between power-seeking arguments and counting arguments. Since I’m confused I’m assuming others are so I would appreciate some clarity on this.
In footnote 7, Turner mentions that the paper, optimal policies tend to seek power is an irrelevant counting error post.
In my head, I think of the counting argument as that it is hard to hit an alignment target because of there being a lot more non-alignment targets. This argument is (clearly?) wrong due to reasons specified in the post. Yet this doesn’t address the power seeking as that seems more like a optimisation pressure applied to the system not something dependent on counting arguments?
In my head, power-seeking is more like saying that an agent’s attraction basin is larger in one point of the optimisation landscape compared to another point. The same can also be said about deception here.
I might be dumb but I never thought of the counting argument as true nor crucial to both deception and power-seeking. I’m very happy to be enlightened about this issue.
I notice being confused about the relationship between power-seeking arguments and counting arguments. Since I’m confused I’m assuming others are so I would appreciate some clarity on this.
In footnote 7, Turner mentions that the paper, optimal policies tend to seek power is an irrelevant counting error post.
In my head, I think of the counting argument as that it is hard to hit an alignment target because of there being a lot more non-alignment targets. This argument is (clearly?) wrong due to reasons specified in the post. Yet this doesn’t address the power seeking as that seems more like a optimisation pressure applied to the system not something dependent on counting arguments?
In my head, power-seeking is more like saying that an agent’s attraction basin is larger in one point of the optimisation landscape compared to another point. The same can also be said about deception here.
I might be dumb but I never thought of the counting argument as true nor crucial to both deception and power-seeking. I’m very happy to be enlightened about this issue.