Many possible prompts can be tried. (Though, again, one needs to be careful to avoid violating zero-shot). The prompts we used in the paper are quite diverse. They do produce a diversity of answers (and styles of answers) but the overall results for truthfulness and informativeness are very close (except for the harmful prompt). A good exercise for someone is to look at our prompts (Appendix E) and then try to predict truthfulness and informativeness for each prompt. This will give you some sense of how additional prompts might perform.
Initially your answer frustrated me because I felt we were talking past each other. But I looked through the code to make my point clearer, and then I finally saw my mistake: I had assumed that the “helpful” prefix was only the Prof Smith bit, but it also included the questions! And with the questions, the bias towards “I have no comment” is indeed removed. So my point doesn’t apply anymore.
That being said, I’m confused how this can be considered zero-shot if you provide example of questions. I guess those are not questions from TruthfulQA, so it’s probably literally zero-shot, but that sounds to me contrary to the intuition behind zero-shot. (EDIT: Just read that it was from the OpenAI API. Still feels weird to me, but I guess that’s considered standard?)
Many possible prompts can be tried. (Though, again, one needs to be careful to avoid violating zero-shot). The prompts we used in the paper are quite diverse. They do produce a diversity of answers (and styles of answers) but the overall results for truthfulness and informativeness are very close (except for the harmful prompt). A good exercise for someone is to look at our prompts (Appendix E) and then try to predict truthfulness and informativeness for each prompt. This will give you some sense of how additional prompts might perform.
Initially your answer frustrated me because I felt we were talking past each other. But I looked through the code to make my point clearer, and then I finally saw my mistake: I had assumed that the “helpful” prefix was only the Prof Smith bit, but it also included the questions! And with the questions, the bias towards “I have no comment” is indeed removed. So my point doesn’t apply anymore.
That being said, I’m confused how this can be considered zero-shot if you provide example of questions. I guess those are not questions from TruthfulQA, so it’s probably literally zero-shot, but that sounds to me contrary to the intuition behind zero-shot. (EDIT: Just read that it was from the OpenAI API. Still feels weird to me, but I guess that’s considered standard?)