Ng and LeCun on the 6-Month Pause (Transcript)

This is a transcript a brief chat between Andrew Ng and Yann LeCun on the prospect of the 6-Month Pause. They are both highly skeptical of the pause and more generally of “AI doomerism”. Yann also briefly touches on why he thinks LLMs won’t scale to AGI.

Full video can be found here. Transcript was made using Whisper. Extensive editing was then done for readability.

Ng:
I’m Andrew Ng, founder of DeepLearning.ai and I’m delighted to have you join me here today and with me also is Yann LeCun who is VP and Chief AI Scientist of Meta, a professor at NYU and Turing Award winner.

So Yann and I have been friends for a long time and both he and I have thought at length about this six-month moratorium proposal and felt it was an important enough topic that Yann and I felt like we want to chat about it with you here today.

LeCun:
Hi Andrew, it’s a pleasure to be here.

Ng:
So just to summarize the situation, I think over the last decade or maybe even over the last longer 20-30 years we’ve seen very exciting progress in AI. Deep learning is working well and then even in the last one or two years feels like maybe there’s even a further acceleration of progress in AI with generative AI systems such as ChatGPT, LLAMA, also image generation, mid-journey, stable diffusion, DALL-E, feels like maybe AI has gotten even faster and associated with that there are people that worry about AI, you know, fairness, bias, social economic displacement. There are also the further out speculative worries about AGI, evil sentient killer robots, but I think that there are real worries about harms, possible real harms today and possibly other harms in the future that people worry about. So in this environment, the Future Life Institute put out a call for a six-month pause, or moratorium, on training AI models that are even more powerful than OpenAI’s GPT-4 model and a number of people, very smart people, including our friend Yoshua Bengio, Stuart Russell, Elon Musk, signed on to this proposal. I think Jan and I are both concerned about this proposal, but Yann, why don’t you start? Do you want to share your take on this proposal?

LeCun:
Well, my first reaction to this is that calling for a delay in research and development smacks me of a new wave of obscurantism, essentially, like why slow down the progress of knowledge and science? Then there is the question of products, like, you know, I’m all for regulating products that get in the hands of people. I don’t see the point of regulating research and development. I don’t think this serves any purpose other than reducing the knowledge that we could use to actually make, you know, technology better, safer also.

Ng:
I feel like while AI today has some risks of harm. Bias, fairness, concentration of power, those are real issues.

I think that it’s also creating tremendous value, right? I think with deep learning over the last 10 years and even the last year or so, last many months, the number of generative AI ideas and how to use it for education or healthcare, responsible coaching, is incredibly exciting. The value that so many people are creating to help other people using AI, and I think that as amazing as GPT-4 is today, building an even better than GPT-4 will help all of these applications, will help a lot of people. So pausing that progress seems like it would create a lot of harm and slow down the creation of very valuable stuff that will help a lot of people.

LeCun:
Right, and you know, I think there is probably several motivations from the various signatories of that letter. Some of them are perhaps on one extreme, are worried about, you know, AGI being turned on at one point and then eliminating humanity on a short notice. I think few people really believe in this kind of scenario or believe it’s a, you know, definite threat that is not, that cannot be stopped. Then there are people I think are much more reasonable who think that there is real potential harms and danger that needs to be dealt with, and I agree with them. There is a lot of issues with AI, making AI systems controllable, making them factual if they are supposed to provide information, etc. Making them non-toxic. And I think there, there is a bit of a lack of imagination in the sense of it’s not like future AI systems would be designed on the same blueprint as current autoregressive elements like ChatGPT and GPT-4 or other systems before them like Galactica or BARD or, you know, whatever. I think there’s going to be new ideas that are going to make those systems much more controllable. And so the problem of, then it becomes a problem of designing objective for those systems that are aligned with human values and policies. And I think, you know, thinking that somehow we’re smart enough to build those systems to be super intelligent and not smart enough to design good objectives so that they behave properly, I think is a very, very strong assumption that is, it’s just not, it’s very, it’s very low probability. And then, you know, there is the question of, which is more kind of a political question really, is the question of what impact on people and the economy will be due to the fact that, you know, those products at the moment are produced by a very small number of companies that are going to gain power and influence and are motivated by their profit motive. Does that have, you know, intrinsic risks? And the answer to this is proper regulation, obviously, but not stopping R&D, it’s regulating products.

Ng:
You know, about aligning AI, I have mixed feelings about that term, but getting AI to behave well, it’s actually been interesting to see the rapid progress in last year, it’s acknowledging the real problems of, you know, AI-generated toxic speech, those are real problems. But I feel like not everyone appreciates when we move from base language models like GPT-3 to the instruction two models like the Chat GPT or GPT-3.5-turbo or whatever, that was real progress. I think the models are now much less toxic, far from perfect, and they still make stuff up. But I’m seeing real progress with instruction two models, and this is why many companies are shifting in that direction. But I think that stepping on the gas and doubling down on AI safety and aligning AI, that feels more constructive than proposing a blanket pause.

LeCun:
Yeah, I agree.

Ng:
Yeah. And actually, you’re using the term AI doomers, I don’t think I’ve, that’s an interesting term. Actually, I feel like you, Yann, for a long time, you’ve been speaking up against AI hype. You know, I think when deep learning was new, a lot of people had unrealistic expectations about what it could and couldn’t do. And frankly, I think I was over optimistic about self-driving cars too, so I made that mistake. But a lot of well-meaning people, you know, just overestimated, right, what it could and could not do, and unfortunately contributed a little bit to hype, and you and I have been speaking against hype. And I think this type of AI doom saying, or AI doom as your terminology, I think is actually another type of hype, that AI could escape and overpower us all. And that type of AI doom saying hype is, I think, also harmful, because it also creates unrealistic expectations.

LeCun:
I agree. And I think also the fact that now, you know, things like chatGPT and GPT-4 have been in the hands of people, and then the, you know, the Microsoft version of them, that people have been playing with it, it gives the impression, perhaps, that we are closer to human level intelligence. Because we, as humans, are very, we’re sort of very linguist, you know, language oriented. We think that when something is fluent, it’s also intelligent, but it’s not true. And those systems have a very superficial understanding of reality. They don’t have any experience of reality, really. They’re trained purely from text. I mean, GPT-4 is trained a little bit with images as well, but mostly, their understanding of the world is extremely superficial. And this is one reason why they can essentially produce nonsense that sounds convincing, but isn’t. And we’re not that close to human level intelligence. There is no question in my mind, and this is not hype, there is no question in my mind that sometimes in the next few decades, we’ll have systems whose intelligence level equals or surpasses human intelligence in all the domains where humans are intelligent. But human intelligence is not intelligent. Human intelligence is very specialized. We think we have general intelligence, but we don’t. We are incredibly specialized. And those systems are going to be much better than us in all kinds of domains. There’s no question this will happen. It’s not going to happen tomorrow. And until we have some sort of blueprint of a system that has at least a chance of reaching human intelligence, discussions on how to properly make them safe and all that is premature, because how can you, I don’t know, design seatbelts for a car if the car doesn’t exist? How can you design safe jet engines if you haven’t invented the airplane yet? So I think some of those questions are premature, and I think a bit of the sort of panic towards that future is misguided.

Ng:
So we’ve made one year of wildly exciting progress in AI in the last one year, and hopefully we’ll keep on making wildly exciting progress the next 30 or 50 years, and then maybe we’ll have some sort of AGI. But until we’re closer, it’s really difficult to know how that will go. And then as we’re on that 50-year journey to get to AGI it’s difficult to see how a half-month pause before the next 49 and a half years of work is particularly helpful.

LeCun:
Yeah, I mean, clearly we’re not anywhere close to human level intelligence, otherwise we would have level 5 autonomous driving, as you were mentioning, and we don’t, right? How is it that a teenager can learn to drive in about 20 hours of training? And, you know, we don’t have self-driving cars. And we have systems that are fluent in language, but the amount of data they’ve been trained on, which is on the order of a trillion words, it would take on the order of 22,000 years for a human reading eight hours a day to go through this. I mean, that’s just insane, right? So clearly the kind of intelligence that is formed by this is not the type that we observe in humans. If we did, I mean, it’s another example of the Moravec paradox, things that appear sophisticated to us in terms of intelligence, like playing chess and writing text, turn out to be relatively simple for machines, whereas things we take for granted, which you know, that any 10-year-old can do, I don’t know, clearing out the table, the dinner table, and, you know, filling up the dishwasher. We don’t have robots that can do this yet. You worked on this, Andrew, I remember, on filling up dishwashers…

Ng:
Yes, it was over optimistic.

So I think you agree, right? We’re very far from AGI.

So in addition to the problems with the premises of AGI Escape, one of the challenges with the proposal is it doesn’t seem implementable. I feel like some things aren’t implementable. So, for example, you know, I think that having, proposing that we do more to research AI safety, you know, maybe more transparency, auditing, let’s have more public, right, NSF or other public funding for basic research on AI. Those would be constructive proposals.

But the idea of asking AI labs to slow down just doesn’t seem practical or implementable to me. Especially in this frankly competitive business environment with labs and countries trying to build advanced technologies and creating a lot of value. And the only thing worse than that would be if government steps in to pass legislations to pause AI, which would be really terrible innovation policy. I can’t imagine it being a good idea for government to pass laws to slow down progress of technology that even the governments, and frankly, even that don’t fully understand.

LeCun:
And when we’re talking about GPT4 or GPT or whatever OpenAI puts out at the moment, we’re not talking about research and development.

We’re talking about product development.

So OpenAI kind of pivoted from an AI research lab that was relatively open, as the name indicates, into a for-profit company and now a kind of a contract research lab,mostly for Microsoft, that is product development and doesn’t reveal anything about the product anymore, about how they work. So this is product development. It’s not AI R&D. And stopping the product development, I think is a question. You want to perhaps regulate products that are made available to the public if they endanger public safety.

Obviously, that’s where government should intervene. That’s what it does for drugs and airplanes and cars and just about everything that consumers can put out. So some level of regulation here, if it’s done right, is not necessarily a bad thing. But you don’t regulate the technology. You don’t regulate R&D in a technology.

I’ll give you an example. I’m going to give you an example.

Also, you have to be very careful if you regulate technology that people want and helps. There was knee jerk reactions of similar types when the printing press started popping up, the Catholic Church was extremely resistant to it because it said it would destroy society. And it did. Because it basically enabled people to read the Bible. And it created the Protestant movement and also had bad side effects, like religious wars in Europe for a couple of hundred years. But it also enabled the dissemination of the Enlightenment and science and rationalism and democracy, which resulted in the creation of the United States, by the way. So this had overall a good effect. So what we need to do when a new technology is put in place like this is make sure the benefits, the positive effects, are maximized and the negative ones are minimized. But that doesn’t necessarily go through stopping it. The Ottoman Empire, when the printing press appeared, stopped it because they were concerned that, again, this would get people to think outside of the religious orthodoxy or their political power. And that part of the world basically became an intellectual backwater, even though at the time they were pretty dominant in mathematics and science.

By stopping a system that amplifies people’s intelligence, which is what we’re talking about here, you’re taking a step back in terms of progress of humanity. I mean, AI is going to be an amplification of human intelligence. We might see a new renaissance because of it, a new enlightenment, if you want. And why would we want to stop that?

Ng:
Yeah, and just to add to the idea of regulating technology, I agree with you almost entirely. And there is just one exception.

So a lot of the authors of the petition referred to the Asilomar 1975 Conference on Recombinant DNA. And I feel like that analogy is not a great analogy, in my opinion, because when researchers are doing research on DNA, recombinant DNA, back in the 60s and 70s, there are research on monkey viruses, for example. And there was a real risk of creating a new disease that would escape and infect people and harm. And as we’ve just seen with COVID, pandemics are a real thing. And they’re really, really terrible for society. And so I think back in 1975, the Asilomar conference put in place containment mechanisms for doing certain types of DNA research. And I think that was a good idea. And the reason I find it troubling to make an analogy between that Asilomar conference and what happens in AI, which seems to be one of the popular themes out there, AI escape is just not. I don’t see any realistic risk of AI escape, unlike escape of infectious diseases. That is a risk. But AI escape would imply not only do we get to AGI, which will take decades probably, but that AGI is so wily and so smart, it outsmarts all of these billions of people that don’t want AI to harm us or kill us. And that’s just an implausible scenario for decades or maybe centuries or maybe even longer.

LeCun:
I think there are interesting intellectual debates to have around this question. Some of the ones that you see online are unreasonable, but some of them are reasonable.

There’s been a discussion following an article that I co-wrote with Tony Zador that was published in Scientific American four years ago, whose title was Don’t Fear the Terminator. And basically what we said, those scenarios where AI systems will want to dominate humanity are unrealistic because you need to have a motivation to dominate to actually dominate. And that motivation exists in humans because we are a social species. It exists in a number of other social species. It does not exist in non-social species because they don’t have the need for it. And there is no reason it will exist in AI systems that we design. We can design their objectives so that they are non-dominant, submissive, or they obey certain rules that are in line with the best interests of humanity as a whole. So as you said, this scenario is completely implausible.

Ng:
One thing that grips me is I think societal sentiment toward tech. I feel like 10, 20 years ago, tech was all good. It’s wonderful, treating everyone. And then I think over the last five, six, seven years, society realized that there were real problems with tech. I feel like the media sentiment swung appropriately, but swung too far in the direction of viewing tech as bad.

And I feel we need a more balanced view. Let’s acknowledge the wonderful things tech is doing and treating for everyone, even while acknowledging the very real risk of harm. We do have problems with safety, interoperability, transparency, concentration of power, privacy. Those are real things that we’re almost vehemently working on. And then I think this media swing a little bit too far to a negative sentiment on tech. I think feeds into this hype and sentiment that I don’t think is helpful. I think actually a more balanced view would be more helpful for moving forward.

LeCun:
There’s this kind of perception.

Working at Meta and being a bit of a public figure, I’ve been a bit of the center of that controversy somehow, even though I’m not involved in Meta’s content policy or anything like that, or privacy policy or anything.

But what I think people don’t realize is that it’s not just there is a problem with tech that the public and the press has identified, but the tech industry itself has identified those problems too and has attempted to correct them and mitigate them.

So the side effects of ranking algorithms in social networks, for example, some of it was predictable and fixed before it happened, and some of it wasn’t. It was fixed when it occurred. In the meantime, there was perhaps bad side effects. And some of them are limited by technology. So one thing that people don’t realize is that progress in AI is part of the solution to some of the problems that social networks have encountered. And I’m not just talking about Facebook or Instagram. That includes YouTube and TikTok and Twitter and everything. So the progress in AI has allowed things like taking down hate speech more efficiently and things like that. And this is due entirely to large language models.

Because we have large language models, not of the same type, not the autoregressive type that we see in chat GPT and others, but the sort of BERT style, we can do things like do a pretty good job, a better job than we ever could in detecting hate speech in most languages in the world. That was impossible before. That only occurred in the last four or five years. That’s due to progress in AI. AI is part of the solution there. It’s not the problem. It’s part of the solution.

Ng:
Yeah, that’s a good point. And I feel like the way that there are irresponsible ways to roll products, like rolling a self-driving car that kills people, that would be irresponsible. Well, there are irresponsible ways to do that anyway. But then I think a lot of technology, it just isn’t perfect the first time. And the practical way to roll it out is to roll it out in a self-contained way, limit harm, limit dangers, and incrementally roll it out so we can better monitor and mitigate the harmful cases, and then develop the solutions to it. So the pausing progress seems like it doesn’t seem hopeful. We should release things in a very controlled way and make sure it’s not harmful. And I think some of the risks of harm are overblown.

Audience Question:
What are the conditions/scenarios/tipping point that, if it happened, the AI pause would be a good idea?

LeCun:
I think if there is something that is deployed, perhaps, on a small scale for trials, and there is identification of real harm, because there are two kinds of harm.

There is potential harm and real harm. There is imagined harm.

And if there is real harm you should stop that product from being deployed. Now, should it mean that you should ban all of AI research? No. So it’s very difficult to identify before it occurs. But when it occurs, on a small scale, you take corrective measures. And that’s what’s happened in the history of technology. I mean, the first cars were very unsafe. They didn’t have good brakes. They didn’t have safety belts. There were no traffic signals, blah, blah, blah. And those things were put in place progressively to make it safer, same for aviation. And eventually, you have a regulatory agency that makes sure those products are safe. So the way it’s going to work, there is no intrinsic qualitative difference between AI and previous technological advances.

Ng:
Yeah, and I’ll add to that. In the case of genetic pathogen research, there was a plausible path to harm, pathogens escaping, recombinant DNA as low in confidence. That was very difficult to pull back. Even the most powerful governments in the world can’t shut off a pathogen once it’s released. And I think the fact that today, governments can pass legislation to cause companies to shut down their servers, we do have the option to shut things down relatively quickly if it causes harm. So that is another strong defense.

Audience Question:
How should we interpret leading experts’ disagreement on pause, e.g. Yoshua Bengio being a signatory, while Yann opposes it?

LeCun:
Yoshua is an old friend, a very good friend. And we’ve known each other since he was a master’s student and I was a postdoc. So that goes back a very long time to the 1980s. And people have different opinions and different motivations.

He certainly has slightly different political opinions than mine. I think he’s motivated by those beliefs signing this.

He sees the danger of companies controlling a technology for profit motive as intrinsically bad. I don’t believe so. I think it should be controlled, but I don’t think it’s intrinsically bad necessarily. He is very much against the idea that R&D and AI should be secret, done in secret. And I agree with him. I’m a very, very strong proponent of open research. But again, we’re not talking about research here. We’re talking about products. I’ve been a huge proponent of open research. And I think it should continue that way. Partly, I think people are unhappy about open AI being secretive now because most of the ideas that have been used by them in their products were actually not from them. They were ideas that were published by people at Google and FAIR and various other academic groups et cetera. And now they are being under lock and key. But this is not going to last very long in the sense that there’s going to be a lot of other products that have very similar capabilities, if not better, within relatively short order. Yeah, open AI has a bit of an advance because of the flywheel of data of having many users for the system that allows them to fine tune it. But it’s not going to last.

Audience Question: Are you concerned that in the near future, most advanced AI models we own solely by a few companies can afford the compute costs?

LeCun:
No.

Ng:
I am concerned a little bit, but not very concerned because the infrastructure layer of LLMs looks hypercompetitive right now. I think open AI did a great job with scaling up LLMs, transformers, and really pushing forward the instruction to models. Really, at least two major breakthroughs I would attribute to open AI. But Meta’s release of LLAMA was a great thing. I think that Google’s PaLM model is becoming hypercompetitive with many large and small organizations releasing different LLMs. So right now, it looks very competitive. And in the market space, I see a lot of excitement then building the application layer on top of the large LLMs.

LeCun:
Absolutely. I think this is going to get democratized actually fairly quickly. So yes, if you want to train the biggest LLM ever, you’re going to need a lot of computing resources. But most usage, I mean, there’s going to be some sort of pyramid of all kinds of models where the simplest models, which are still useful, are going to be more widely available. You’re going to be able to run them on relatively modest hardware, possibly on mobile devices on short order. And you’re going to have a lot of such LLMs or systems of that type available with different degrees of openness for either research groups or products on short order. It’s competitive, which means there is a lot of motivation for people to put things out there. And some of them are going to be more open than others.

Audience Question: How do you think about the role of AI in engineering? What should a student do to prepare for the future and make good engineering applications in the future?

Ng:
So I think AI is wildly exciting. AI is a general purpose technology, both deep learning, supervised learning, and generative AI. These are general purpose technologies, meaning they aren’t good just for one thing. They’re good for a lot of different things. And so these technological advancements give students opportunities to go around, tech, non-tech, all corners of the economy, to find valuable applications, be it education or health care or coaching or lots of things, right, improving industrial automation, whatever, to find important applications and go and build them in a responsible, ethical, fair way to create value for everyone.

So it’s a fantastic time to learn about these technologies and go find use cases and build, and not to pause this exciting building.

LeCun:
Right. So that’s on the side of engineering and product development. But I think it’s also on the side of AI and machine learning and machine learning. And I think that’s a big challenge. Nobody should get the idea that AI is solved. As I said, we still don’t have domestic robots that can clear the table and fill up the dishwasher. We’re missing something big in terms of the learning capabilities of humans and animals that we still cannot reproduce in machines. They can’t really plan at all. They can produce plans that already exist, that they stored in their memory, but they can’t really plan. So there’s a big, big challenge for the next few years in AI research for machines that can achieve human intelligence. And it’s not going to be obtained by simply scaling up autoregressive LLMs and training them with tokenized multimodal data. It’s going to take a lot more than that.

Ng and LeCun on the 6-Month Pause (Transcript)

Audience Question: What are the conditions/​scenarios/​tipping point that, if it happened, the AI pause would be a good idea?

Audience Question: How should we interpret leading experts’ disagreement on pause, e.g. Yoshua Bengio being a signatory, while Yann opposes it?

Audience Question: Are you concerned that in the near future, most advanced AI models we own solely by a few companies can afford the compute costs?

Audience Question: How do you think about the role of AI in engineering? What should a student do to prepare for the future and make good engineering applications in the future?

Audience Question:
What are the conditions/scenarios/tipping point that, if it happened, the AI pause would be a good idea?

Audience Question:
How should we interpret leading experts’ disagreement on pause, e.g. Yoshua Bengio being a signatory, while Yann opposes it?