No it’s not, and obviously so. The actual topic is AI safety. It’s not false vacuum, it’s not a black marble, or a marble of any color for that matter.
It is, and Connor said so repeatedly throughout the conversation. AI safety is a subtopic, a special case, of Connor’s main thrust, albeit the most important one. (Machine transcript, emphasis mine.)
The world is not ergodic, actually. It’s actually a very non-ergodic you can die. [...] I’m wondering if you agree with this, forget [A]I for a moment that at some point not saying it’s [A]I just at some point we will develop technology that is so powerful that if you fuck it up, it blows up everybody.
The way I see things is, is that never mind. Like, I know AGI is the topic I talk about the most and whatever comes the most pressing one, but [A]I actually AGI is not the main thing I care about. The main thing I care about is technology in general, and of which AGI is just the most salient example in the current future. You know, 50 if I was born 50 years ago, I would care about nukes [...] And the thing I fundamentally care about is the stewardship of technology. [...] of course things can go bad. It’s like we’re[...] mimetically engineering, genetically engineering, super beings. Like, of course this is dangerous. Like, if we were genetically engineering super tigers, people would be like, hey, that seems maybe a bit, but let let’s talk about this
Beff starts talking before he could finish, so skipping ahead a bit:
The way I see things is, is that our civilization is just not able to handle powerful technology. I just don’t trust our institutions. Our leaders are, you know, distributed systems. Anything with, you know, hyper powerful technology at this point in time, this doesn’t mean we couldn’t get to systems that could handle this technology without catastrophic or at least vastly undesirable side effects. But I don’t think we’re there.
But I want to make clear again, just the point I’m trying to make here. Is that the point I’m trying to make here is, is that predictably, if you have a civilization that doesn’t even try, that just accelerates fast as possible, predictably guaranteed, you’re not going to make it. You’re definitely not going to make it. At some point, you will develop technology that is too powerful to handle if you just have the hands of random people, and if you do it as unsafe as possible, eventually an accident will happen. We almost nuked ourselves twice during the Cold War, where only a single person was between a nuke firing and it not happening. If the same thing happens with, say, superintelligence or some other extremely powerful technology which will happen in your scenario sooner or later. You know, maybe it goes well for 100 years, maybe it goes well for a thousand years, but eventually your civilization is just not going to make it.
It is, and Connor said so repeatedly throughout the conversation. AI safety is a subtopic, a special case, of Connor’s main thrust, albeit the most important one. (Machine transcript, emphasis mine.)
Non-ergodicity, not necessarily AI:
Connor explicitly calls out AGI as not his main point:
Beff starts talking before he could finish, so skipping ahead a bit:
This is Connor’s mindset in the whole debate. Backing up a bit:
Also the rolling death comment I mentioned previously. And the comment about crazy wackos.