If humans are supposed to be able to detect things going wrong and shut things down, that requires that they are exposed to the unencrypted feed. At this point, the humans are the weakest link, not the encryption. Similar for anything else external that you need / want AI to access while it’s being trained and tested.
Edited to add: particularly if we are talking about not some theoretical sensible humans, but about real humans that started with “do not worry about LLMs, they are not agentic”, and then promptly connected LLMs to agentic APIs.
I agree with you. Humans are the weakest link. This scheme isn’t able to solve that. It is meant to be a marginal improvement over a physical kill switch and make theft and release of the model more difficult.
If humans are supposed to be able to detect things going wrong and shut things down, that requires that they are exposed to the unencrypted feed. At this point, the humans are the weakest link, not the encryption. Similar for anything else external that you need / want AI to access while it’s being trained and tested.
Edited to add: particularly if we are talking about not some theoretical sensible humans, but about real humans that started with “do not worry about LLMs, they are not agentic”, and then promptly connected LLMs to agentic APIs.
I agree with you. Humans are the weakest link. This scheme isn’t able to solve that. It is meant to be a marginal improvement over a physical kill switch and make theft and release of the model more difficult.