Here’s a sketch for what I’d like to see in the future—a better version of the scenario experiment done above:
2-4 people sit down for a few hours together.
For the first 1-3 hours, they each write a Scenario depicting their ‘median future’ or maybe ‘modal future.’ The scenarios are written similarly to the one I wrote above, with dated ‘stages.’ The scenarios finish with superintelligence, or else it-being-clear-superintelligence-is-many-decades-away-at-least.
As they write, they also read over each other’s scenarios and ask clarifying questions. E.g. “You say that in 2025 they can code well but unreliably—what do you mean exactly? How much does it improve the productivity of, say, OpenAI engineers?”
By the end of the period, the scenarios are finished & everyone knows roughly what each stage means because they’ve been able to ask clarifying questions.
Then for the next hour or so, they each give credences for each stage of each scenario. Credences in something like “ASI by year X” where X is the year ASI happens in the scenario.
They also of course discuss and critique each other’s credences, and revise their own.
At the end, hopefully some interesting movements will have happened in people’s mental models and credences, and hopefully some interesting cruxes will have surfaced—e.g. it’ll be more clear what kinds of evidence would actually cause timelines updates, were they to be observed.
The scenarios, credences, and maybe a transcript of the discussion then gets edited and published.
Here’s a sketch for what I’d like to see in the future—a better version of the scenario experiment done above:
2-4 people sit down for a few hours together.
For the first 1-3 hours, they each write a Scenario depicting their ‘median future’ or maybe ‘modal future.’ The scenarios are written similarly to the one I wrote above, with dated ‘stages.’ The scenarios finish with superintelligence, or else it-being-clear-superintelligence-is-many-decades-away-at-least.
As they write, they also read over each other’s scenarios and ask clarifying questions. E.g. “You say that in 2025 they can code well but unreliably—what do you mean exactly? How much does it improve the productivity of, say, OpenAI engineers?”
By the end of the period, the scenarios are finished & everyone knows roughly what each stage means because they’ve been able to ask clarifying questions.
Then for the next hour or so, they each give credences for each stage of each scenario. Credences in something like “ASI by year X” where X is the year ASI happens in the scenario.
They also of course discuss and critique each other’s credences, and revise their own.
At the end, hopefully some interesting movements will have happened in people’s mental models and credences, and hopefully some interesting cruxes will have surfaced—e.g. it’ll be more clear what kinds of evidence would actually cause timelines updates, were they to be observed.
The scenarios, credences, and maybe a transcript of the discussion then gets edited and published.