May I suggest that for an organization which really could create superhuman general AI, more important than a vague vow to go slow and be careful, would be a concrete vow to try to solve the problem of friendliness/safety/alignment. (And I’ll promote June Ku’s https://metaethical.ai as the best model we have of what that would look like. Not yet practical, but a step in the right direction.)
Perhaps it is a result of my being located on the margins, but I have never seen slowing the overall AI race as a tactic worth thinking about. That it is already an out-of-control race in which no one even knows all the competitors, has been my model for a long time. I also think one should not proceed on the assumption that there are years (let alone decades) to spare. For all we know, a critical corner could be turned at any moment.
Perhaps it feels different if you’re DeepMind or MIRI; you may feel that you know most of the competitors, and have some chance of influencing them in a collegial way. But was OpenAI’s announcement of GPT-3 in 2020, a different kind of escalation than DeepMind’s announcement of AlphaGo in 2016? OpenAI has been concerned with AI safety from the beginning; they had Paul Christiano on their alignment team for years.
I am not exactly here to say that DeepMind is that much better! :) One thing I dislike about the OP is that it makes it seem like the problem is specifically with OpenAI compared to other companies. If OpenAI came first and then Elon went and founded DeepMind that would approximately just as bad, or even slightly worse.
I agree that maybe an arms race was inevitable, in which case founding OpenAI maybe wasn’t a bad thing after all. Maybe. But maybe not.
It’s true that OpenAI had some great safety researchers. Now most of them have quit. (There are still some that remain). But they probably could have got jobs at DeepMind, so this isn’t relevant to evaluating Elon’s decision.
Also, there’s the whole openness ideal/norm. Terrible idea, for reasons various people (e.g. Scott Alexander) said at the time. (I can try to remember what the post was called if you like… it made the same point as Yudkowsky here, if we haven’t solved alignment yet and we give AI to everyone then we are killing ourselves. If we have solved alignment, great, but that’s the difficult part and we haven’t done that yet. That point and a few others.)
For the sake of discussion, let’s suppose that the next big escalation in AI power is the final one, and that it’s less than five years away. Any thoughts on what the best course of action is?
Warm take: Our main hope lies in the EA/longtermist/AI-safety community. I say this because my first-order answer to your question is “no idea,” so instead I go meta and say “Rally the community. Get organized. Create common knowledge. Get the wisest people in a room together to discuss the problem and make a plan. Everybody stick to the plan.” (The plan will of course be a living document that updates over time in response to new info. The point is that for a crisis, you need organization; you need a chain of command. It seems to be the main way that humans are able to get large numbers of people working effectively together on short notice. The main problem with organizations is that they tend to become corrupt and decay over time, hence the importance of competition/markets/independence. However this is less of a problem on short timescales and anyway what choice do we have?)
We have good-enough alignment for the AIs we have. We don’t have a general solution to alignment that will work for the ASIs we don’t have. We also don’t know whether we we need one, ie. we don’t know that we need to solve ASI alignment beyond getting ASIs to work acceptably.
I constantly see conflations of AI and ASI. It doesn’t give me much faith in amateur (unrelated to industry) efforts at AI safety.
May I suggest that for an organization which really could create superhuman general AI, more important than a vague vow to go slow and be careful, would be a concrete vow to try to solve the problem of friendliness/safety/alignment. (And I’ll promote June Ku’s https://metaethical.ai as the best model we have of what that would look like. Not yet practical, but a step in the right direction.)
Perhaps it is a result of my being located on the margins, but I have never seen slowing the overall AI race as a tactic worth thinking about. That it is already an out-of-control race in which no one even knows all the competitors, has been my model for a long time. I also think one should not proceed on the assumption that there are years (let alone decades) to spare. For all we know, a critical corner could be turned at any moment.
Perhaps it feels different if you’re DeepMind or MIRI; you may feel that you know most of the competitors, and have some chance of influencing them in a collegial way. But was OpenAI’s announcement of GPT-3 in 2020, a different kind of escalation than DeepMind’s announcement of AlphaGo in 2016? OpenAI has been concerned with AI safety from the beginning; they had Paul Christiano on their alignment team for years.
I am not exactly here to say that DeepMind is that much better! :) One thing I dislike about the OP is that it makes it seem like the problem is specifically with OpenAI compared to other companies. If OpenAI came first and then Elon went and founded DeepMind that would approximately just as bad, or even slightly worse.
I agree that maybe an arms race was inevitable, in which case founding OpenAI maybe wasn’t a bad thing after all. Maybe. But maybe not.
It’s true that OpenAI had some great safety researchers. Now most of them have quit. (There are still some that remain). But they probably could have got jobs at DeepMind, so this isn’t relevant to evaluating Elon’s decision.
Also, there’s the whole openness ideal/norm. Terrible idea, for reasons various people (e.g. Scott Alexander) said at the time. (I can try to remember what the post was called if you like… it made the same point as Yudkowsky here, if we haven’t solved alignment yet and we give AI to everyone then we are killing ourselves. If we have solved alignment, great, but that’s the difficult part and we haven’t done that yet. That point and a few others.)
For the sake of discussion, let’s suppose that the next big escalation in AI power is the final one, and that it’s less than five years away. Any thoughts on what the best course of action is?
Warm take: Our main hope lies in the EA/longtermist/AI-safety community. I say this because my first-order answer to your question is “no idea,” so instead I go meta and say “Rally the community. Get organized. Create common knowledge. Get the wisest people in a room together to discuss the problem and make a plan. Everybody stick to the plan.” (The plan will of course be a living document that updates over time in response to new info. The point is that for a crisis, you need organization; you need a chain of command. It seems to be the main way that humans are able to get large numbers of people working effectively together on short notice. The main problem with organizations is that they tend to become corrupt and decay over time, hence the importance of competition/markets/independence. However this is less of a problem on short timescales and anyway what choice do we have?)
https://slatestarcodex.com/2015/12/17/should-ai-be-open/
We have good-enough alignment for the AIs we have. We don’t have a general solution to alignment that will work for the ASIs we don’t have. We also don’t know whether we we need one, ie. we don’t know that we need to solve ASI alignment beyond getting ASIs to work acceptably.
I constantly see conflations of AI and ASI. It doesn’t give me much faith in amateur (unrelated to industry) efforts at AI safety.