There are many ways for something way smarter than a human to take control and do whatever the heck it wants. Focusing on making just one of them harder just give you a false sense of security. From Relatively Reasonable AI NotKillEveryonism Takes:
Hack most computers and internet-enabled things at once, analyze all the info, scale up, make or steal a lot of money, use to [let’s say bribe, hire and blackmail] many people, have your humans use your resources to take over, have them build your robots, your robots kill them.
If your response is ‘you can’t hack all those things’ then I give up, what does smarter even mean to you. If you think people wouldn’t let the rest of it play out, they’d grow spines and fight back and not dig their own graves, read more history.
I mean, no, the AI won’t do it that way, they’ll do something way faster and safer and smarter, I would do something smarter and I’m way dumber than the AI by assumption. Obviously the smarter-than-human AIs would think of new things and build new tech.
To be honest, I don’t believe this story the way he tells it and I don’t expect many people outside our community would be persuaded. To be clear, there are versions of this story I can believe, but I haven’t heard anyone tell it in a persuasive way.
(Actually, scratch that: I think Hollywood already told this story, several times, usually without nanotech being a crux, and quite persuasively. I think if you ask regular people, their objection to the possibility of the robopocalypse is usually long timelines and not the fundamental problem of humans losing control. In fact I think most people, even techno-optimists, agree that we are doomed to lose control.)
I think I should have said “lose control eventually.” I’m becoming more optimistic that AIs are easy to align. Maybe you can get GPT-4 to say the n-word with an optimized prompt, but for normal usage, it’s not exactly a 4channer.
Focusing on making just one of them harder just give you a false sense of security.
I think this is a bad mindset. It’s a fully general argument. The Swiss Cheese Model would be a much better approach than “we have to find the one perfect solution and ignore all other solutions.” To be blunt, I think that the alignment community makes the perfect the enemy of the good.
There are many ways for something way smarter than a human to take control and do whatever the heck it wants. Focusing on making just one of them harder just give you a false sense of security. From Relatively Reasonable AI NotKillEveryonism Takes:
To be honest, I don’t believe this story the way he tells it and I don’t expect many people outside our community would be persuaded. To be clear, there are versions of this story I can believe, but I haven’t heard anyone tell it in a persuasive way.
(Actually, scratch that: I think Hollywood already told this story, several times, usually without nanotech being a crux, and quite persuasively. I think if you ask regular people, their objection to the possibility of the robopocalypse is usually long timelines and not the fundamental problem of humans losing control. In fact I think most people, even techno-optimists, agree that we are doomed to lose control.)
Well, if you agree that humans are bound to lose control, what do you disagree with?
I think I should have said “lose control eventually.” I’m becoming more optimistic that AIs are easy to align. Maybe you can get GPT-4 to say the n-word with an optimized prompt, but for normal usage, it’s not exactly a 4channer.
Another point:
I think this is a bad mindset. It’s a fully general argument. The Swiss Cheese Model would be a much better approach than “we have to find the one perfect solution and ignore all other solutions.” To be blunt, I think that the alignment community makes the perfect the enemy of the good.