It sounds like we might not disagree a lot? We’ve exchanged a few responses to each other in the past that may have given the impression that we disagree strongly on AI timelines, but plausibly we just frame things differently.
Roughly speaking, when I say “AI timelines” I’m referring to the time during which AI fundamentally transforms the world, not necessarily when “an AGI” is built somewhere. I think this framing is more useful because it tracks more closely what EAs actually care about when they talk about AI.
I also don’t think that the moment GDP accelerates is the best moment to intervene, though my framing here is different. I’d be inclined to discard a binary model of intervention during which all efforts after some critical threshold are wasted. Rather, intervention is a lot like the time-value of money in finance. In general, it’s better to have money now rather than later; similarly, it’s better to intervene earlier rather than later. But the value of interventions diminish continuously as time goes onwards, eventually getting near zero.
The best way to intervene also depends a lot on how far we are from AI-induced growth. So, for example, it might not be worth trying to align current algorithms, because that sort of work will be relatively more useful when we know which algorithms are actually being used to build advanced AI. Relatively speaking, it might be worth more right now to build institutional resilience, in the sense of creating incentive structures for actors to care about alignment. And hypothetically, if I knew about the AI alignment problem in, say, the 1920s, I might have recommended investing in the stock market until we have a better sense as to what form AI will take.
The two posts I linked above explain my view on what EAs should care about for timelines; it’s pretty similar to yours. I call it AI-PONR, but basically it just means “a chunk of time where the value of interventions drops precipitously, to a level significantly below its present value, such that when we make our plans for how to use our money, our social capital, our research time, etc. we should basically plan to have accomplished what we want to have accomplished by then.” Things that could cause AI-PONR: An AI takes over the world. Persuasion tools destroy collective epistemology. AI R&D tools make it so easy to build WMD’s that we get a vulnerable world. Etc. Note that I disagree that the time when AI fundamentally transforms the world is what we care about, because I think AI-PONR will come before that point. (By fundamentally transforms the world, do you mean something notably different from “accelerates GDP?”) I’d be interested to hear your thoughts on this framework, since it seems you’ve been thinking along similar lines and might have more expertise than me with the background concepts from economics.
So it sounds like we do disagree on something substantive, and it’s how early in takeoff AI-PONR happens. And/or what timelines look like. I think there’s, like, a 25% chance that nanobots will be disassembling large parts of Earth by 2030, but I think that the 2030′s will look exactly as you predict up until it’s too late.
Makes sense. I also agree that this is what the 2030′s will look like; I don’t expect GDP growth to accelerate until it’s already too late.
The quest for testable-prior-to-AI-PONR predictions continues...
It sounds like we might not disagree a lot? We’ve exchanged a few responses to each other in the past that may have given the impression that we disagree strongly on AI timelines, but plausibly we just frame things differently.
Roughly speaking, when I say “AI timelines” I’m referring to the time during which AI fundamentally transforms the world, not necessarily when “an AGI” is built somewhere. I think this framing is more useful because it tracks more closely what EAs actually care about when they talk about AI.
I also don’t think that the moment GDP accelerates is the best moment to intervene, though my framing here is different. I’d be inclined to discard a binary model of intervention during which all efforts after some critical threshold are wasted. Rather, intervention is a lot like the time-value of money in finance. In general, it’s better to have money now rather than later; similarly, it’s better to intervene earlier rather than later. But the value of interventions diminish continuously as time goes onwards, eventually getting near zero.
The best way to intervene also depends a lot on how far we are from AI-induced growth. So, for example, it might not be worth trying to align current algorithms, because that sort of work will be relatively more useful when we know which algorithms are actually being used to build advanced AI. Relatively speaking, it might be worth more right now to build institutional resilience, in the sense of creating incentive structures for actors to care about alignment. And hypothetically, if I knew about the AI alignment problem in, say, the 1920s, I might have recommended investing in the stock market until we have a better sense as to what form AI will take.
The two posts I linked above explain my view on what EAs should care about for timelines; it’s pretty similar to yours. I call it AI-PONR, but basically it just means “a chunk of time where the value of interventions drops precipitously, to a level significantly below its present value, such that when we make our plans for how to use our money, our social capital, our research time, etc. we should basically plan to have accomplished what we want to have accomplished by then.” Things that could cause AI-PONR: An AI takes over the world. Persuasion tools destroy collective epistemology. AI R&D tools make it so easy to build WMD’s that we get a vulnerable world. Etc. Note that I disagree that the time when AI fundamentally transforms the world is what we care about, because I think AI-PONR will come before that point. (By fundamentally transforms the world, do you mean something notably different from “accelerates GDP?”) I’d be interested to hear your thoughts on this framework, since it seems you’ve been thinking along similar lines and might have more expertise than me with the background concepts from economics.
So it sounds like we do disagree on something substantive, and it’s how early in takeoff AI-PONR happens. And/or what timelines look like. I think there’s, like, a 25% chance that nanobots will be disassembling large parts of Earth by 2030, but I think that the 2030′s will look exactly as you predict up until it’s too late.