This is an unordered list of uncertainties about the future of AI, trying to be comprehensive– trying to include everything reasonably decision-relevant and important/tractable.
This list is mostly written from a forecasting perspective. A useful complementary perspective to forecasting would be strategy or affordances or what actors can do and what they should do. This list is also written from a nontechnical perspective.
Timelines
Capabilities as a function of inputs (or input requirements for AI of a particular capability level)
Spending by leading labs
Cost of compute
Ideas and algorithmic progress
Endogeneity in AI capabilities
What would be good? Interventions?
Takeoff (speed and dynamics)
dcapabilities/dinputs (or returns on cognitive reinvestment or intelligence explosion or fast recursive self-improvement): Is there a threshold of capabilities such that self-improvement or other progress is much greater slightly above that point than slightly below it? (If so, where is it?) Will there be a system that can quickly and cheaply improve itself (or create a more capable successor), such that the improvements enable similarly large improvements, and so on until the system is much more capable? Will a small increase in inputs cause a large increase in capabilities (like the difference between chimps and humans) (and if so, around human-level capabilities or where)? How fast will progress be?
Will dcapabilities/dinputs be very high because of recursive self-improvement?
Will dcapabilities/dinputs be very high because of generality being important and monolithic/threshold-y?
Will dcapabilities/dinputs be very high because ideas are discrete (and in particular, will there be a single “secret sauce” idea)? [Seems intractable and unlikely to be very important.]
dimpacts/dcapabilities: Will small increases in capabilities (on the order of the variance between different humans) cause large increases in impacts?
Qualitatively, how will AI research capabilities affect AI progress; what will AI progress look like when AI research capabilities are a big deal?
One dimension or implication: will takeoff be local or nonlocal? How distributed will it be; will it look like recursive self-improvement or the industrial revolution?
What does this depend on?
Endogeneity in AI capabilities: how do AI capabilities affect AI capabilities? (Potential alternative framing: dinputs/dtime.)
How (much) will AI tools accelerate research?
Will AI labs generate substantial revenue?
Will AI systems make AI seem more exciting or frightening? What effect would this have on AI progress?
What other endogeneities exist?
Weak AI
How will the strategic landscape be different in the future?
What would a powerful, unaligned AI agent do? [Answer: the details are unpredictable, but the high-level outcome is pretty clear; it would very likely be catastrophic. (But note there is reasonable disagreement, e.g..)]
Technical problems around AI systems doing what their controllers want [doesn’t fit into this list well]
How to make powerful AI systems aligned to human preferences
Or: how to make powerful AI systems that do not cause a catastrophe
How to make powerful AI systems interpretable to human understanding
Are general systems more powerful than similar narrow tools?
Are agents more powerful than similar tools?
Does generality appear by default in capable systems? (This relates to takeoff.)
Does agency (or goal-directedness or consequentialism or farsightedness) appear by default in capable systems?
AI labs’ behavior and racing for AI
How will labs think about AI?
What actions could labs perform; what are they likely to do by default?
What would it be better if labs did; what interventions are tractable?
States’ behavior
How will states think about AI?
What actions could states perform? What are they likely to do by default?
What would it be better if states did; what interventions are tractable?
Public opinion
How the public thinks about AI and framing; what the public thinks about AI, what memes would spread widely and how that depends on other facts about the world; how all of that translates into attitudes and policy preferences
Wakeup to capabilities
Wakeup to alignment risk and warning shots for alignment
What (facts about public opinion) would be good? Interventions?
Paths to powerful AI (relates to timelines, AI risk modes, and more)
How successful will reinforcement-learning agents built on large language models be?
Meta level. To carve nature at its joints, we must [use good nodes / identify the true nodes]. A node is [good insofar as / true if] its causes and effects are modular, or we can losslessly compress phenomena related to it into effects on it and effects from it.
“The cost of compute” is an example of a great node (in the context of the future of AI): it’s affected by various things (choices made by Nvidia, innovation, etc.), and it affects various things (capability-level of systems made by OpenAI, relative importance of money vs talent at AI labs, etc.), and we lose nothing by thinking in terms of the cost of compute (relative to, e.g., the effects of the choices made by Nvidia on the capability-level of systems made by OpenAI).
“When Moore’s law will end” is an example of something that is not a node (in the context of the future of AI), since you’d be much better off thinking in terms of the underlying causes and effects.
The relations relevant to nodes are analytical not causal. For example, “the cost of compute” is a node between “evidence about historical progress” and “timelines,” not just between “stuff Nvidia does” and “stuff OpenAI does.” (You could also make a causal model, but here I’m interested in analytical models.)
Object level. I’m not sure how good “timelines,” “takeoff,” “polarity,” and “wakeup to capabilities” are as nodes. Most of the time it seems fine to talk about e.g. “effects on timelines” and “implications of timelines.” But maybe this conceals confusion.
List of uncertainties about the future of AI
This is an unordered list of uncertainties about the future of AI, trying to be comprehensive– trying to include everything reasonably decision-relevant and important/tractable.
This list is mostly written from a forecasting perspective. A useful complementary perspective to forecasting would be strategy or affordances or what actors can do and what they should do. This list is also written from a nontechnical perspective.
Timelines
Capabilities as a function of inputs (or input requirements for AI of a particular capability level)
Spending by leading labs
Cost of compute
Ideas and algorithmic progress
Endogeneity in AI capabilities
What would be good? Interventions?
Takeoff (speed and dynamics)
dcapabilities/dinputs (or returns on cognitive reinvestment or intelligence explosion or fast recursive self-improvement): Is there a threshold of capabilities such that self-improvement or other progress is much greater slightly above that point than slightly below it? (If so, where is it?) Will there be a system that can quickly and cheaply improve itself (or create a more capable successor), such that the improvements enable similarly large improvements, and so on until the system is much more capable? Will a small increase in inputs cause a large increase in capabilities (like the difference between chimps and humans) (and if so, around human-level capabilities or where)? How fast will progress be?
Will dcapabilities/dinputs be very high because of recursive self-improvement?
Will dcapabilities/dinputs be very high because of generality being important and monolithic/threshold-y?
Will dcapabilities/dinputsbe very high because ideas are discrete (and in particular, will there be a single “secret sauce” idea)?[Seems intractable and unlikely to be very important.]dimpacts/dcapabilities: Will small increases in capabilities (on the order of the variance between different humans) cause large increases in impacts?
Related: payoff thresholds and human-competition threshold
Qualitatively, how will AI research capabilities affect AI progress; what will AI progress look like when AI research capabilities are a big deal?
One dimension or implication: will takeoff be local or nonlocal? How distributed will it be; will it look like recursive self-improvement or the industrial revolution?
What does this depend on?
Endogeneity in AI capabilities: how do AI capabilities affect AI capabilities? (Potential alternative framing: dinputs/dtime.)
How (much) will AI tools accelerate research?
Will AI labs generate substantial revenue?
Will AI systems make AI seem more exciting or frightening? What effect would this have on AI progress?
What other endogeneities exist?
Weak AI
How will the strategic landscape be different in the future?
Due to weak AI?
Due to other factors? (See Relevant pre-AGI possibilities.)
Will weak AI make AI risk clearer, and especially make it more legible (relates to “warning shots”)?
Will AI progress cause substantial misuse or conflict? What would that look like?
Will AI progress enable pivotal acts or processes?
Misalignment risk sources/modes
What misalignment would occur by default, and how would it be bad? Possible (overlapping) scenarios include inner alignment failure, outer alignment failure, getting what you measure, influence-seeking, more Paul Christiano stories, multipolar failure, a treacherous turn, a sharp left turn (or distributional leap), and more.
What would a powerful, unaligned AI agent do?[Answer: the details are unpredictable, but the high-level outcome is pretty clear; it would very likely be catastrophic. (But note there is reasonable disagreement, e.g..)]Technical problems around AI systems doing what their controllers want[doesn’t fit into this list well]How to make powerful AI systems aligned to human preferencesOr: how to make powerful AI systems that do not cause a catastropheHow to make powerful AI systems interpretable to human understandingHow to solve problems arounddecision theory and strategic interaction(and whether they’re important)How to solveWei-Dai-style philosophy problems(and whether they’re important)How to solve problems arounddelegation involving multiple humans or multiple AI systems(and whether they’re important)Polarity (relates to takeoff, endogeneity, and timelines)
What determines or affects polarity?
What are the effects or implications of polarity on alignment and stabilization?
What are the effects or implications of polarity on what the long-term future looks like, conditional on achieving alignment and stabilization?
What would be good? Interventions?
Proximate and ultimate uses of powerful AI
What uses of powerful AI would be great? How good would various possible uses of powerful AI be?
Conditional on achieving alignment, what’s likely to occur (and what could foresighted actors cause to occur or not to occur)?
Agents vs tools and general systems vs narrow tools (relates to tool AI and Comprehensive AI Services)
Are general systems more powerful than similar narrow tools?
Are agents more powerful than similar tools?
Does generality appear by default in capable systems? (This relates to takeoff.)
Does agency (or goal-directedness or consequentialism or farsightedness) appear by default in capable systems?
AI labs’ behavior and racing for AI
How will labs think about AI?
What actions could labs perform; what are they likely to do by default?
What would it be better if labs did; what interventions are tractable?
States’ behavior
How will states think about AI?
What actions could states perform? What are they likely to do by default?
What would it be better if states did; what interventions are tractable?
Public opinion
How the public thinks about AI and framing; what the public thinks about AI, what memes would spread widely and how that depends on other facts about the world; how all of that translates into attitudes and policy preferences
Wakeup to capabilities
Wakeup to alignment risk and warning shots for alignment
What (facts about public opinion) would be good? Interventions?
Paths to powerful AI (relates to timelines, AI risk modes, and more)
How successful will reinforcement-learning agents built on large language models be?
How successful will comprehensive AI services be?
How successful will language model bureaucracies be?
How successful will STEM AI or Skunkworks-style AI be?
How successful will simulating evolution be?
How successful will whole-brain emulation or neuromorphic AI be?
When is thinking in terms of paths or roadmaps useful? What other high-level paths are relevant?
Meta and miscellanea
Epistemic stuff
Research methodology and organization: how to do research and organize researchers
Forecasting methodology: how to do forecasting better
Collective epistemology: how to share and aggregate beliefs and knowledge
Decision theory
Simulation hypothesis
Do we live in a simulation? (If so, what’s the deal?)
What should we do if we live in a simulation (as a function of what the deal is)?
Movement/community/field stuff
Maybe this list would be more useful if it had more pointers to relevant work?
Maybe this list would be more useful if it included stuff that’s important that I don’t feel uncertain about? But probably not much of that exists?
I like lists/trees/graphs. I like the ideas behind Clarifying some key hypotheses in AI alignment and Modelling Transformative AI Risks. Perhaps this list is part of the beginning of a tree/graph for AI forecasting not including alignment stuff.
Meta level. To carve nature at its joints, we must [use good nodes / identify the true nodes]. A node is [good insofar as / true if] its causes and effects are modular, or we can losslessly compress phenomena related to it into effects on it and effects from it.
“The cost of compute” is an example of a great node (in the context of the future of AI): it’s affected by various things (choices made by Nvidia, innovation, etc.), and it affects various things (capability-level of systems made by OpenAI, relative importance of money vs talent at AI labs, etc.), and we lose nothing by thinking in terms of the cost of compute (relative to, e.g., the effects of the choices made by Nvidia on the capability-level of systems made by OpenAI).
“When Moore’s law will end” is an example of something that is not a node (in the context of the future of AI), since you’d be much better off thinking in terms of the underlying causes and effects.
The relations relevant to nodes are analytical not causal. For example, “the cost of compute” is a node between “evidence about historical progress” and “timelines,” not just between “stuff Nvidia does” and “stuff OpenAI does.” (You could also make a causal model, but here I’m interested in analytical models.)
Object level. I’m not sure how good “timelines,” “takeoff,” “polarity,” and “wakeup to capabilities” are as nodes. Most of the time it seems fine to talk about e.g. “effects on timelines” and “implications of timelines.” But maybe this conceals confusion.