We’re quite lucky that labs are building AI in pretty much the same way:
same paradigm (deep learning)
same architecture (transformer plus tweaks)
same dataset (entire internet text)
same loss (cross entropy)
same application (chatbot for the public)
Kids, I remember when people built models for different applications, with different architectures, different datasets, different loss functions, etc. And they say that once upon a time different paradigms co-existed — symbolic, deep learning, evolutionary, and more!
This sameness has two advantages:
Firstly, it correlates catastrophe. If you have four labs doing the same thing, then we’ll go extinct if that one thing is sufficiently dangerous. But if the four labs are doing four different things, then we’ll go extinct if any of those four things are sufficiently dangerous, which is more likely.
It helps ai safety researchers because they only need to study one thing, not a dozen. For example, mech interp is lucky that everyone is using transformers. It’d be much harder to do mech interp if people were using LSTMs, RNNs, CNNs, SVMs, etc. And imagine how much harder mech interp would be if some labs were using deep learning, and others were using symbolic ai!
Implications:
One downside of closed research is it decorrelates the activity of the labs.
I’m more worried by Deepmind than Meta, xAI, Anthropic, or OpenAI. Their research seems less correlated with the other labs, so even though they’re further behind than Anthropic or OpenAI, they contribute more counterfactual risk.
I was worried when Elon announced xAI, because he implied it was gonna be a stem ai (e.g. he wanted it to prove Riemann Hypothesis). This unique application would’ve resulted in a unique design, contributing decorrelated risk. Luckily, xAI switched to building AI in the same way as the other labs — the only difference is Elon wants less “woke” stuff.
We’re quite lucky that labs are building AI in pretty much the same way:
same paradigm (deep learning)
same architecture (transformer plus tweaks)
same dataset (entire internet text)
same loss (cross entropy)
same application (chatbot for the public)
Kids, I remember when people built models for different applications, with different architectures, different datasets, different loss functions, etc. And they say that once upon a time different paradigms co-existed — symbolic, deep learning, evolutionary, and more!
This sameness has two advantages:
Firstly, it correlates catastrophe. If you have four labs doing the same thing, then we’ll go extinct if that one thing is sufficiently dangerous. But if the four labs are doing four different things, then we’ll go extinct if any of those four things are sufficiently dangerous, which is more likely.
It helps ai safety researchers because they only need to study one thing, not a dozen. For example, mech interp is lucky that everyone is using transformers. It’d be much harder to do mech interp if people were using LSTMs, RNNs, CNNs, SVMs, etc. And imagine how much harder mech interp would be if some labs were using deep learning, and others were using symbolic ai!
Implications:
One downside of closed research is it decorrelates the activity of the labs.
I’m more worried by Deepmind than Meta, xAI, Anthropic, or OpenAI. Their research seems less correlated with the other labs, so even though they’re further behind than Anthropic or OpenAI, they contribute more counterfactual risk.
I was worried when Elon announced xAI, because he implied it was gonna be a stem ai (e.g. he wanted it to prove Riemann Hypothesis). This unique application would’ve resulted in a unique design, contributing decorrelated risk. Luckily, xAI switched to building AI in the same way as the other labs — the only difference is Elon wants less “woke” stuff.
Let me know if I’m thinking about this all wrong.