Monitoring surveillance in order to see if anyone is breaking rules seems to be quite a bounded task, and in fact is one that we are already in the process of automating (using our current AI systems, which are basically all bounded).
Of course, there are lots of other tasks that are not as clear. But to the extent that you believe the Factored Cognition hypothesis, you should believe that we can make bounded services that nevertheless do a very good job.
Monitoring surveillance in order to see if anyone is breaking rules seems to be quite a bounded task, and in fact is one that we are already in the process of automating (using our current AI systems, which are basically all bounded).
That seems true, but if this surveillance monitoring isn’t 100% effective, won’t you still need an agential police to deal with any threats that manage to evade the surveillance? Or do you buy Eric’s argument that we can use a period of “unopposed preparation” to make sure that the defense, even though it’s bounded, is still much more capable than any agential threat it might face?
Sorry, when I said “there are lots of other tasks that are not as clear”, I meant that there are a lot of other tasks relevant to policing and security that are not as clear, such as police to deal with threats that evade surveillance. I think the optimism here comes from our ability to decompose tasks, such that we can take a task that seems to require goal-directed agency (like “be the police”) and turn it into a bunch of subtasks that no longer look agential.
I agree that in the long term, agent AI could probably improve faster than CAIS, but I think CAIS could still be a solution.
Regardless of how it is aligned, aligned AI will tend to improve slower than unaligned AI, because it is trying to achieve a more complicated goal, human oversight takes time, etc. To prevent unaligned AI, aligned AI will need a head start, so it can stop any unaligned AI while it’s still much weaker. I don’t think CAIS is fundamentally different in that respect.
If the reasoning in the post that CAIS will develop before AGI holds up, then CAIS would actually have an advantage, because it would be easier to get a head start.
This seems likely to me as well, especially since “service” is by definition bounded and agent is not.
Monitoring surveillance in order to see if anyone is breaking rules seems to be quite a bounded task, and in fact is one that we are already in the process of automating (using our current AI systems, which are basically all bounded).
Of course, there are lots of other tasks that are not as clear. But to the extent that you believe the Factored Cognition hypothesis, you should believe that we can make bounded services that nevertheless do a very good job.
That seems true, but if this surveillance monitoring isn’t 100% effective, won’t you still need an agential police to deal with any threats that manage to evade the surveillance? Or do you buy Eric’s argument that we can use a period of “unopposed preparation” to make sure that the defense, even though it’s bounded, is still much more capable than any agential threat it might face?
Sorry, when I said “there are lots of other tasks that are not as clear”, I meant that there are a lot of other tasks relevant to policing and security that are not as clear, such as police to deal with threats that evade surveillance. I think the optimism here comes from our ability to decompose tasks, such that we can take a task that seems to require goal-directed agency (like “be the police”) and turn it into a bunch of subtasks that no longer look agential.
I agree that in the long term, agent AI could probably improve faster than CAIS, but I think CAIS could still be a solution.
Regardless of how it is aligned, aligned AI will tend to improve slower than unaligned AI, because it is trying to achieve a more complicated goal, human oversight takes time, etc. To prevent unaligned AI, aligned AI will need a head start, so it can stop any unaligned AI while it’s still much weaker. I don’t think CAIS is fundamentally different in that respect.
If the reasoning in the post that CAIS will develop before AGI holds up, then CAIS would actually have an advantage, because it would be easier to get a head start.