I do agree this would be bad if secret and non-privacy preserving.
I think we need to separate out use cases. For reporting on bad behavior (but not quite bad enough to trigger red flags and account closure), then a system like Clio seems ideal.
But what if the model is uncertain about something and wants to ask for clarification? But the user isn’t being bad, just posing a difficult question.
In such a case, as a user, I’d want the model to explicitly ask me for permission to ask the Anthropic staff its question, and for me to need to approve the exact wording. There’s plenty of situations that could be great in!
I do agree this would be bad if secret and non-privacy preserving.
I think we need to separate out use cases. For reporting on bad behavior (but not quite bad enough to trigger red flags and account closure), then a system like Clio seems ideal.
But what if the model is uncertain about something and wants to ask for clarification? But the user isn’t being bad, just posing a difficult question.
In such a case, as a user, I’d want the model to explicitly ask me for permission to ask the Anthropic staff its question, and for me to need to approve the exact wording. There’s plenty of situations that could be great in!