And the thing is, most of the things that have become dangerous when connected to the web have become dangerous when human hackers discovered novel uses for them—IoT light bulbs notably (yes, these light bulb actual harm as the drivers of DoS attacks etc). And the dangers of just statically exploitable systems have increased over time as ill-intentioned humans learn more misuses of them. Moreover, such uses include immediate bad-acting as well as cobbling together a fully bad-aligned system (adding invisible statefullness for example). And LLM seems inherently insecure on a wholly different level than an OS, database or etc—an LLM’s behavior is fundamentally unspecified.
And the thing is, most of the things that have become dangerous when connected to the web have become dangerous when human hackers discovered novel uses for them—IoT light bulbs notably (yes, these light bulb actual harm as the drivers of DoS attacks etc). And the dangers of just statically exploitable systems have increased over time as ill-intentioned humans learn more misuses of them. Moreover, such uses include immediate bad-acting as well as cobbling together a fully bad-aligned system (adding invisible statefullness for example). And LLM seems inherently insecure on a wholly different level than an OS, database or etc—an LLM’s behavior is fundamentally unspecified.