I want to explicitly mark this (in my mind) as “babble” of the sort that is “Hufflepuff Bones” from HPMOR. I don’t want to have this relationship with the leadership at AI labs. I’m way more interested in finding positive sum actions to take (e.g. trades that AI labs are actively happy about) than adversarial moves.
I want them to stop lighting enormous fires next to open fuel repositories, which is unfortunately what these “leaders” are currently paid millions of dollars to do. Given that, I’m not sure what option there is. They are already adversaries. Nonprofit leadership probably doesn’t have much of a vested interest and we should definitely talk to them, but these guys? Seriously, what strategy remains?
Man, wtf is this argument? Yes you should talk to leaders in industry. I have not yet tried at all hard enough on the “work together” strategy to set it on fire at this moment. I don’t think such people they see themselves as adversaries, I don’t think we have been acting like adversaries, and I think this has allowed us to have a fair amount of conversation and cooperation with them.
I feel a bit like you’re saying “a new country is stockpiling nukes, so let’s make sure to quicklystop talking to them”. We’re in a free market economy, everyone is incentivized to build these companies, not just the people leading existing ones. That’s like most of the problem.
I think it’s good and healthy to think about your BATNA and figure out your negotiating position, so thinking about this seems good to me, but just because you might be able to exercise unilateral control doesn’t mean you should, it lowers trust and the ability for everyone to work together on anything.
I’m not confident here and maybe I should already have given up after OpenAI was founded but I’m not ready to call everyone adversaries, it’s pretty damn hard to backtrack on that, and it makes conversation and coordination way way harder.
I think my problem is that I sometimes use “moral culpability” as some sort of proxy for “potential for positive outcomes following dialogue”. Should reiterate that it was always my opinion that we should be doing more outreach to industry leaders, even if my hopes are low, especially if it turns out we haven’t really tried it.
Edit: After further thought I also think the frustration I have with this attitude is:
We’re not going to convince everybody.
Wild success means diverting significant but not necessarily critical amounts of resources (human, monetary, etc.) going toward AI capabilities research toward other less dangerous things.
Less AI capabilities research dries up the very short term money. Someone from #1 who we can’t convince, or just doesn’t care, is going to be mad about this.
So it’s my intuition that, if you’re not willing to annoy e.g. DeepMind’s executive leadership, you are basically unable to commit to any strategy with a chance of working. It sucks too because this is the type of project where one bad organization will still end up killing everybody else, eventually. But this is the problem that must be solved, and it involves being willing to piss some people off.
I want to explicitly mark this (in my mind) as “babble” of the sort that is “Hufflepuff Bones” from HPMOR. I don’t want to have this relationship with the leadership at AI labs. I’m way more interested in finding positive sum actions to take (e.g. trades that AI labs are actively happy about) than adversarial moves.
I want them to stop lighting enormous fires next to open fuel repositories, which is unfortunately what these “leaders” are currently paid millions of dollars to do. Given that, I’m not sure what option there is. They are already adversaries. Nonprofit leadership probably doesn’t have much of a vested interest and we should definitely talk to them, but these guys? Seriously, what strategy remains?
Man, wtf is this argument? Yes you should talk to leaders in industry. I have not yet tried at all hard enough on the “work together” strategy to set it on fire at this moment. I don’t think such people they see themselves as adversaries, I don’t think we have been acting like adversaries, and I think this has allowed us to have a fair amount of conversation and cooperation with them.
I feel a bit like you’re saying “a new country is stockpiling nukes, so let’s make sure to quickly stop talking to them”. We’re in a free market economy, everyone is incentivized to build these companies, not just the people leading existing ones. That’s like most of the problem.
I think it’s good and healthy to think about your BATNA and figure out your negotiating position, so thinking about this seems good to me, but just because you might be able to exercise unilateral control doesn’t mean you should, it lowers trust and the ability for everyone to work together on anything.
I’m not confident here and maybe I should already have given up after OpenAI was founded but I’m not ready to call everyone adversaries, it’s pretty damn hard to backtrack on that, and it makes conversation and coordination way way harder.
I think my problem is that I sometimes use “moral culpability” as some sort of proxy for “potential for positive outcomes following dialogue”. Should reiterate that it was always my opinion that we should be doing more outreach to industry leaders, even if my hopes are low, especially if it turns out we haven’t really tried it.
Edit: After further thought I also think the frustration I have with this attitude is:
We’re not going to convince everybody.
Wild success means diverting significant but not necessarily critical amounts of resources (human, monetary, etc.) going toward AI capabilities research toward other less dangerous things.
Less AI capabilities research dries up the very short term money. Someone from #1 who we can’t convince, or just doesn’t care, is going to be mad about this.
So it’s my intuition that, if you’re not willing to annoy e.g. DeepMind’s executive leadership, you are basically unable to commit to any strategy with a chance of working. It sucks too because this is the type of project where one bad organization will still end up killing everybody else, eventually. But this is the problem that must be solved, and it involves being willing to piss some people off.
I am not sure what the right stance is here, and your points seem reasonable. (I am willing to piss people off.)