I still like & stand by this post. I refer back to it constantly. It does two things:
1. Argue that an AI-induced point of no return could significantly before, or significantly after, world GDP growth accelerates—and indeed will probably come before!
2. Argue that we shouldn’t define timelines and takeoff speeds in terms of economic growth. So, against “is there a 4 year doubling before a 1 year doubling?” and against “When will we have TAI = AI capable of doubling the economy in 4 years if deployed?”
I think both things are pretty important; I think focus on GWP is distracting us from the metrics that really matter and hence hindering epistemic progress, and I think that most of the AI risk comes from scenarios in which AI-PONR happens before GWP accelerates, so it’s important to evaluate the plausibility of such scenarios.
I talked with Paul about this post once and he said he still wasn’t convinced, he still expects GWP to accelerate before the point of no return. He said some things that I found helpful (e.g. gave some examples of how AI tech will have dramatically shorter product development cycles than historical products, such that you really will be able to deploy it and accelerate the economy in the months to years before substantially better versions are created), but nothing that significantly changed my position either. I would LOVE to see more engagement/discussion of this stuff. (I recognize Paul is busy etc. but lots of people (most people?) have similar views, so there should be plenty of people capable of arguing for his side. On my side, there’s MIRI, see this comment, which is great and if I revise this post I’ll want to incorporate some of the ideas from it. Of course the best thing to incorporate would be good objections & replies, hence why I wish I had some. I’ve at least got the previously-mentioned one from Paul. Oh, and Paul also had an objection to my historical precedent which I take seriously.)
(I am the author)
I still like & stand by this post. I refer back to it constantly. It does two things:
1. Argue that an AI-induced point of no return could significantly before, or significantly after, world GDP growth accelerates—and indeed will probably come before!
2. Argue that we shouldn’t define timelines and takeoff speeds in terms of economic growth. So, against “is there a 4 year doubling before a 1 year doubling?” and against “When will we have TAI = AI capable of doubling the economy in 4 years if deployed?”
I think both things are pretty important; I think focus on GWP is distracting us from the metrics that really matter and hence hindering epistemic progress, and I think that most of the AI risk comes from scenarios in which AI-PONR happens before GWP accelerates, so it’s important to evaluate the plausibility of such scenarios.
I talked with Paul about this post once and he said he still wasn’t convinced, he still expects GWP to accelerate before the point of no return. He said some things that I found helpful (e.g. gave some examples of how AI tech will have dramatically shorter product development cycles than historical products, such that you really will be able to deploy it and accelerate the economy in the months to years before substantially better versions are created), but nothing that significantly changed my position either. I would LOVE to see more engagement/discussion of this stuff. (I recognize Paul is busy etc. but lots of people (most people?) have similar views, so there should be plenty of people capable of arguing for his side. On my side, there’s MIRI, see this comment, which is great and if I revise this post I’ll want to incorporate some of the ideas from it. Of course the best thing to incorporate would be good objections & replies, hence why I wish I had some. I’ve at least got the previously-mentioned one from Paul. Oh, and Paul also had an objection to my historical precedent which I take seriously.)