I have the opposite intuition regarding economies of scale and CAIS: I feel like it would hold, just to a lesser degree than to a unitary agent. The core of my intuition is that with different optimized AIs, it will be straightforward to determine exactly what the principal-agent problem consists of, and this can be compensated for. I would go as far as to say that such a function seems like a high-likelihood target for monitoring AIs within CAIS, in broadly the same way we can do resource optimization now.
I suspect the limits of both types are probably somewhere north of the current size of the planet’s economy, though.
The core of my intuition is that with different optimized AIs, it will be straightforward to determine exactly what the principal-agent problem consists of, and this can be compensated for.
I feel like it is not too hard to determine principal-agent problems with humans either? It’s just hard to adequately compensate for them.
I partway agree with this: it is much harder to compensate with people than to determine what the problem is.
The reason I still see determining the principal-agent problem as a hard problem with people is that we are highly inconsistent: a single AI is more consistent then a single person, and much more consistent than several people in succession (as is the case with any normal job).
My model for this is that determining what the problem is costs only slightly more for a person than for the AI, but you will have to repeat the process many times for a human position, probably about once per person to fill it.
I see, so the argument is mostly that jobs are performed more stably and so you can learn better how to deal with the principal-agent problems that arise. This seems plausible.
I have the opposite intuition regarding economies of scale and CAIS: I feel like it would hold, just to a lesser degree than to a unitary agent. The core of my intuition is that with different optimized AIs, it will be straightforward to determine exactly what the principal-agent problem consists of, and this can be compensated for. I would go as far as to say that such a function seems like a high-likelihood target for monitoring AIs within CAIS, in broadly the same way we can do resource optimization now.
I suspect the limits of both types are probably somewhere north of the current size of the planet’s economy, though.
I feel like it is not too hard to determine principal-agent problems with humans either? It’s just hard to adequately compensate for them.
I partway agree with this: it is much harder to compensate with people than to determine what the problem is.
The reason I still see determining the principal-agent problem as a hard problem with people is that we are highly inconsistent: a single AI is more consistent then a single person, and much more consistent than several people in succession (as is the case with any normal job).
My model for this is that determining what the problem is costs only slightly more for a person than for the AI, but you will have to repeat the process many times for a human position, probably about once per person to fill it.
I see, so the argument is mostly that jobs are performed more stably and so you can learn better how to deal with the principal-agent problems that arise. This seems plausible.