I’d say my point above would generalize to “there are no strong borders between ‘ordinary language acts’ and ‘genuine hacks’” as far as what level of manipulation ability one can gain over model output. The main further danger would be if the model was given more output channels with which an attacker could work mischief. And that may be appearing as well - notably: https://openai.com/blog/chatgpt-plugins
I’d say my point above would generalize to “there are no strong borders between ‘ordinary language acts’ and ‘genuine hacks’” as far as what level of manipulation ability one can gain over model output. The main further danger would be if the model was given more output channels with which an attacker could work mischief. And that may be appearing as well - notably: https://openai.com/blog/chatgpt-plugins