Proposed Forecasting Technique: Annotate Scenario with Updates (Related to Joe’s Post)
Consider a proposition like “ASI will happen in 2024, not sooner, not later.” It works best if it’s a proposition you assign very low credence to, but that other people you respect assign much higher credence to.
What’s your credence in that proposition?
Step 1: Construct a plausible story of how we could get to ASI in 2024, no sooner, no later. The most plausible story you can think of. Consider a few other ways it could happen too, for completeness, but don’t write them down.
Step 2: Annotate the story with how you think you should update your credence in the proposition, supposing events play out as described in the story. By the end of the story, presumably your credence should be 100% or close to it.
Step 3: Consider whether the pattern of updates makes sense or is irrational in some way. Adjust as needed until you are happy with them.
For example, maybe your first draft has you remain hovering around 1% up until the ASI is discovered to have escaped the lab. But on reflection you should have partially updated earlier, when the lab announced their exciting results, and earlier still when you learned the lab was doing a new kind of AGI project with special sauce X.
You may end up concluding that your current credence should be higher than 1%. But if not, consider the following:
You now have some near-term testable predictions which you are soft-committed to updating on. Unless you’ve bunched all your updates towards the end of the story, if you started with a mere 1%, you gotta do something like 7 doublings of credence in the next 18 months, assuming events play out as they do in the story. Presumably some of those doublings should happen in the next six months or so. So, six months from now, if things have gone more or less as the story indicated, your credence should have at least doubled and probably quadrupled.
You can also ask: Suppose you did this exercise two years ago. Have things gone more or less as the 2024-story you would have made two years ago would have gone? If so, have you in fact updated with 3-5 doublings?
I’m interested to hear feedback on this technique, especially endorsements and critiques.
Proposed Forecasting Technique: Annotate Scenario with Updates (Related to Joe’s Post)
Consider a proposition like “ASI will happen in 2024, not sooner, not later.” It works best if it’s a proposition you assign very low credence to, but that other people you respect assign much higher credence to.
What’s your credence in that proposition?
Step 1: Construct a plausible story of how we could get to ASI in 2024, no sooner, no later. The most plausible story you can think of. Consider a few other ways it could happen too, for completeness, but don’t write them down.
Step 2: Annotate the story with how you think you should update your credence in the proposition, supposing events play out as described in the story. By the end of the story, presumably your credence should be 100% or close to it.
Step 3: Consider whether the pattern of updates makes sense or is irrational in some way. Adjust as needed until you are happy with them.
For example, maybe your first draft has you remain hovering around 1% up until the ASI is discovered to have escaped the lab. But on reflection you should have partially updated earlier, when the lab announced their exciting results, and earlier still when you learned the lab was doing a new kind of AGI project with special sauce X.
You may end up concluding that your current credence should be higher than 1%. But if not, consider the following:
You now have some near-term testable predictions which you are soft-committed to updating on. Unless you’ve bunched all your updates towards the end of the story, if you started with a mere 1%, you gotta do something like 7 doublings of credence in the next 18 months, assuming events play out as they do in the story. Presumably some of those doublings should happen in the next six months or so. So, six months from now, if things have gone more or less as the story indicated, your credence should have at least doubled and probably quadrupled.
You can also ask: Suppose you did this exercise two years ago. Have things gone more or less as the 2024-story you would have made two years ago would have gone? If so, have you in fact updated with 3-5 doublings?
I’m interested to hear feedback on this technique, especially endorsements and critiques.