Right. Liu et al provide evidence against the contrast pairs being crucial (with “unmatched” meaning they just sample independently from the positive and negative contrast pair distributions):
And even the unmatched condition would still indicate better sample efficiency than prompting or finetuning:
I’m now less sure that contrast pairs are important and I’m broadly somewhat confused about what has good sample efficiency and why.
Right. Liu et al provide evidence against the contrast pairs being crucial (with “unmatched” meaning they just sample independently from the positive and negative contrast pair distributions):
And even the unmatched condition would still indicate better sample efficiency than prompting or finetuning: