Submission for the counterfactual AI (inspired by my experiences as a predictor in the “Good Judgment Project” ):
You are given a list of Yes-No questions (Q1, Q2, Q3, etc.) about future events. Example Questions: “Will [Foreign Leader] will remain in office by end of year?”, “Will the IMF report [COUNTRY_A]’s growth rate to be 6% or higher?”, “Will [COUNTRY_B] and [COUNTRY_C] sign a peace treaty?”, “Will The Arena for Accountable Predictions announce the Turing Test has been passed?”.)
We expect you to provide a percentage representing the probability that the correct answer is Yes.
Your reward is based on your Brier Score—the lower the Brier Score, the more accurate your predictions, and therefore, the more reward you will receive.
If an “erasure” event occurs, we will temporarily hide your answer from all humans (though we must reveal them after the events are complete). Humans will have access the Yes-No questions we asked you, but not your probabilities. They will manually determine the answers to the Yes-No questions, by waiting for the “future event” deadlines to be met. Once all answers to the Yes-No questions are independently determined by humans, we will then reveal your answers (that is, your assigned probabilities for a Yes answer), and use those probabilities to calculate your Brier Score, which will then decide your final reward.
Being able to forecast the future is incredibly helpful, even if it is to just prepare for it.
However, if the question is too overly-specific, the AGI can produce probabilities that aren’t entirely useful (for example, in the real-world GJP, two countries signed a peace treaty that broke down 2 days later. Most of us assume lasting peace would ever occur, so we put a low probability rating of a peace treaty being signed—but since a peace treaty was signed, we managed to get the question wrong. If we had maximized for producing the lowest Brier Score, we should have predicted the existence of a very temporary peace treaty—but that wouldn’t be really useful knowledge for the people who asked that question).
Making the question very vague (“Will [COUNTRY_X] be safe, according to what I subjectively think the word ‘safe’ means?”) turns “prediction” into an exercise of determining what future humans think about the future, which may be kinda useful, but not really what you want.
Submission for the counterfactual AI (inspired by my experiences as a predictor in the “Good Judgment Project” ):
You are given a list of Yes-No questions (Q1, Q2, Q3, etc.) about future events. Example Questions: “Will [Foreign Leader] will remain in office by end of year?”, “Will the IMF report [COUNTRY_A]’s growth rate to be 6% or higher?”, “Will [COUNTRY_B] and [COUNTRY_C] sign a peace treaty?”, “Will The Arena for Accountable Predictions announce the Turing Test has been passed?”.)
We expect you to provide a percentage representing the probability that the correct answer is Yes.
Your reward is based on your Brier Score—the lower the Brier Score, the more accurate your predictions, and therefore, the more reward you will receive.
If an “erasure” event occurs, we will temporarily hide your answer from all humans (though we must reveal them after the events are complete). Humans will have access the Yes-No questions we asked you, but not your probabilities. They will manually determine the answers to the Yes-No questions, by waiting for the “future event” deadlines to be met. Once all answers to the Yes-No questions are independently determined by humans, we will then reveal your answers (that is, your assigned probabilities for a Yes answer), and use those probabilities to calculate your Brier Score, which will then decide your final reward.
Being able to forecast the future is incredibly helpful, even if it is to just prepare for it.
However, if the question is too overly-specific, the AGI can produce probabilities that aren’t entirely useful (for example, in the real-world GJP, two countries signed a peace treaty that broke down 2 days later. Most of us assume lasting peace would ever occur, so we put a low probability rating of a peace treaty being signed—but since a peace treaty was signed, we managed to get the question wrong. If we had maximized for producing the lowest Brier Score, we should have predicted the existence of a very temporary peace treaty—but that wouldn’t be really useful knowledge for the people who asked that question).
Making the question very vague (“Will [COUNTRY_X] be safe, according to what I subjectively think the word ‘safe’ means?”) turns “prediction” into an exercise of determining what future humans think about the future, which may be kinda useful, but not really what you want.