Suggested variation, which I’d expect to lead to better results: use raw “completion probabilities” for different answers.E.g. with prompt “Will Russia invade Ukrainian territory in 2022?” extract completion likelihoods of the next few tokes “Yes” and “No”. Normalize.(the prompts you ask for quite unnatural map, at which most untrained humans are pretty bad as well)
Suggested variation, which I’d expect to lead to better results: use raw “completion probabilities” for different answers.
E.g. with prompt “Will Russia invade Ukrainian territory in 2022?” extract completion likelihoods of the next few tokes “Yes” and “No”. Normalize.
(the prompts you ask for quite unnatural map, at which most untrained humans are pretty bad as well)