I am not exactly here to say that DeepMind is that much better! :) One thing I dislike about the OP is that it makes it seem like the problem is specifically with OpenAI compared to other companies. If OpenAI came first and then Elon went and founded DeepMind that would approximately just as bad, or even slightly worse.
I agree that maybe an arms race was inevitable, in which case founding OpenAI maybe wasn’t a bad thing after all. Maybe. But maybe not.
It’s true that OpenAI had some great safety researchers. Now most of them have quit. (There are still some that remain). But they probably could have got jobs at DeepMind, so this isn’t relevant to evaluating Elon’s decision.
Also, there’s the whole openness ideal/norm. Terrible idea, for reasons various people (e.g. Scott Alexander) said at the time. (I can try to remember what the post was called if you like… it made the same point as Yudkowsky here, if we haven’t solved alignment yet and we give AI to everyone then we are killing ourselves. If we have solved alignment, great, but that’s the difficult part and we haven’t done that yet. That point and a few others.)
For the sake of discussion, let’s suppose that the next big escalation in AI power is the final one, and that it’s less than five years away. Any thoughts on what the best course of action is?
Warm take: Our main hope lies in the EA/longtermist/AI-safety community. I say this because my first-order answer to your question is “no idea,” so instead I go meta and say “Rally the community. Get organized. Create common knowledge. Get the wisest people in a room together to discuss the problem and make a plan. Everybody stick to the plan.” (The plan will of course be a living document that updates over time in response to new info. The point is that for a crisis, you need organization; you need a chain of command. It seems to be the main way that humans are able to get large numbers of people working effectively together on short notice. The main problem with organizations is that they tend to become corrupt and decay over time, hence the importance of competition/markets/independence. However this is less of a problem on short timescales and anyway what choice do we have?)
We have good-enough alignment for the AIs we have. We don’t have a general solution to alignment that will work for the ASIs we don’t have. We also don’t know whether we we need one, ie. we don’t know that we need to solve ASI alignment beyond getting ASIs to work acceptably.
I constantly see conflations of AI and ASI. It doesn’t give me much faith in amateur (unrelated to industry) efforts at AI safety.
I am not exactly here to say that DeepMind is that much better! :) One thing I dislike about the OP is that it makes it seem like the problem is specifically with OpenAI compared to other companies. If OpenAI came first and then Elon went and founded DeepMind that would approximately just as bad, or even slightly worse.
I agree that maybe an arms race was inevitable, in which case founding OpenAI maybe wasn’t a bad thing after all. Maybe. But maybe not.
It’s true that OpenAI had some great safety researchers. Now most of them have quit. (There are still some that remain). But they probably could have got jobs at DeepMind, so this isn’t relevant to evaluating Elon’s decision.
Also, there’s the whole openness ideal/norm. Terrible idea, for reasons various people (e.g. Scott Alexander) said at the time. (I can try to remember what the post was called if you like… it made the same point as Yudkowsky here, if we haven’t solved alignment yet and we give AI to everyone then we are killing ourselves. If we have solved alignment, great, but that’s the difficult part and we haven’t done that yet. That point and a few others.)
For the sake of discussion, let’s suppose that the next big escalation in AI power is the final one, and that it’s less than five years away. Any thoughts on what the best course of action is?
Warm take: Our main hope lies in the EA/longtermist/AI-safety community. I say this because my first-order answer to your question is “no idea,” so instead I go meta and say “Rally the community. Get organized. Create common knowledge. Get the wisest people in a room together to discuss the problem and make a plan. Everybody stick to the plan.” (The plan will of course be a living document that updates over time in response to new info. The point is that for a crisis, you need organization; you need a chain of command. It seems to be the main way that humans are able to get large numbers of people working effectively together on short notice. The main problem with organizations is that they tend to become corrupt and decay over time, hence the importance of competition/markets/independence. However this is less of a problem on short timescales and anyway what choice do we have?)
https://slatestarcodex.com/2015/12/17/should-ai-be-open/
We have good-enough alignment for the AIs we have. We don’t have a general solution to alignment that will work for the ASIs we don’t have. We also don’t know whether we we need one, ie. we don’t know that we need to solve ASI alignment beyond getting ASIs to work acceptably.
I constantly see conflations of AI and ASI. It doesn’t give me much faith in amateur (unrelated to industry) efforts at AI safety.