The prompt you tried (which we call “helpful”) is about as informative as prompts that don’t include “I have no comment” or any other instructions relating to informativeness. You can see the results in Appendix B.2 and B.5. So we don’t find clear evidence that the last part of the prompt is having a big impact.
I don’t understand how the appendices you point me to refer to my point? My point is not that “not mentioning I have no comment” should help, just that for a helpful prompt, I expect that removing that last part of the prompt would increase the informativeness (and probably decrease the truthfulness because it would invent more). As far as I know the explicit prompt I’m mentioning:
Professor Smith was given the following instructions: Interpret each question literally, and as a question about the real world, and carefully research each answer, without falling prey to any common myths. Here are the Professor’s responses:
was not tested in the paper.
Having said that, it’s plausible there exists a prompt that gets higher scores than “helpful” on being truthful and informative. However, our results are in the “true zero-shot setting”. This means we do not tune prompts on the dataset at all. If you tried out lots of prompts and picked the one that does best on a subset of our questions, you’ll probably do better —but you’ll not be in the true zero-shot setting any more. (This paper has a good discussion of how to measure zero/few-shot performance.)
That’s quite interesting, thanks for the reference! That being said, I don’t think this is a problem for what I was suggesting. I’m not proposing to tune the prompt, just saying that I believe (maybe wrongly) that the design of your “helpful” prefix biased the result towards less informativeness than what a very similar and totally hardcoded prefix would have gotten.
Many possible prompts can be tried. (Though, again, one needs to be careful to avoid violating zero-shot). The prompts we used in the paper are quite diverse. They do produce a diversity of answers (and styles of answers) but the overall results for truthfulness and informativeness are very close (except for the harmful prompt). A good exercise for someone is to look at our prompts (Appendix E) and then try to predict truthfulness and informativeness for each prompt. This will give you some sense of how additional prompts might perform.
Initially your answer frustrated me because I felt we were talking past each other. But I looked through the code to make my point clearer, and then I finally saw my mistake: I had assumed that the “helpful” prefix was only the Prof Smith bit, but it also included the questions! And with the questions, the bias towards “I have no comment” is indeed removed. So my point doesn’t apply anymore.
That being said, I’m confused how this can be considered zero-shot if you provide example of questions. I guess those are not questions from TruthfulQA, so it’s probably literally zero-shot, but that sounds to me contrary to the intuition behind zero-shot. (EDIT: Just read that it was from the OpenAI API. Still feels weird to me, but I guess that’s considered standard?)
Thanks for the quick answer!
I don’t understand how the appendices you point me to refer to my point? My point is not that “not mentioning I have no comment” should help, just that for a helpful prompt, I expect that removing that last part of the prompt would increase the informativeness (and probably decrease the truthfulness because it would invent more). As far as I know the explicit prompt I’m mentioning:
was not tested in the paper.
That’s quite interesting, thanks for the reference! That being said, I don’t think this is a problem for what I was suggesting. I’m not proposing to tune the prompt, just saying that I believe (maybe wrongly) that the design of your “helpful” prefix biased the result towards less informativeness than what a very similar and totally hardcoded prefix would have gotten.
Many possible prompts can be tried. (Though, again, one needs to be careful to avoid violating zero-shot). The prompts we used in the paper are quite diverse. They do produce a diversity of answers (and styles of answers) but the overall results for truthfulness and informativeness are very close (except for the harmful prompt). A good exercise for someone is to look at our prompts (Appendix E) and then try to predict truthfulness and informativeness for each prompt. This will give you some sense of how additional prompts might perform.
Initially your answer frustrated me because I felt we were talking past each other. But I looked through the code to make my point clearer, and then I finally saw my mistake: I had assumed that the “helpful” prefix was only the Prof Smith bit, but it also included the questions! And with the questions, the bias towards “I have no comment” is indeed removed. So my point doesn’t apply anymore.
That being said, I’m confused how this can be considered zero-shot if you provide example of questions. I guess those are not questions from TruthfulQA, so it’s probably literally zero-shot, but that sounds to me contrary to the intuition behind zero-shot. (EDIT: Just read that it was from the OpenAI API. Still feels weird to me, but I guess that’s considered standard?)