Several people have suggested that a sufficiently smart AI, with the ability to talk to a human as much as it wanted, could persuade the human to “let it out of the box” and give it access to the things it needs to take over the world. This seems plausible to me, say at least 10% probability, which is high enough that it’s worth trying to avoid. And it seems to me that, if you know how to make an AI that’s smart enough to be very useful but will voluntarily restrain itself from persuading humans to hand over the keys to the kingdom, then you must have already solved some of the most difficult parts of alignment. Which means this isn’t a useful intermediate state that can help us reach alignment.
Separately, I’ll mention my opinion that the name of the term “non-violent communication” is either subtle trolling or rank hypocrisy. Because a big chunk of the idea seems to be that you should stick to raw observations and avoid making accusations that would tend to put someone on the defensive… and implying that someone else is committing violence (by communicating in a different style) is one of the most accusatory and putting-them-on-the-defensive things you can do. I’m curious, how many adherents of NVC are aware of this angle on it?
I don’t think NVC tries to put down an opponent, it’s mostly about how you present your ideas. I think it models an opponent as “he tries to win the debate without thinking about my goals. let me think of both mine and theirs goals, so i’m one step ahead”. Which is a bit prerogative and looking down, but not exactly accusatory
Several people have suggested that a sufficiently smart AI, with the ability to talk to a human as much as it wanted, could persuade the human to “let it out of the box” and give it access to the things it needs to take over the world. This seems plausible to me, say at least 10% probability, which is high enough that it’s worth trying to avoid. And it seems to me that, if you know how to make an AI that’s smart enough to be very useful but will voluntarily restrain itself from persuading humans to hand over the keys to the kingdom, then you must have already solved some of the most difficult parts of alignment. Which means this isn’t a useful intermediate state that can help us reach alignment.
Separately, I’ll mention my opinion that the name of the term “non-violent communication” is either subtle trolling or rank hypocrisy. Because a big chunk of the idea seems to be that you should stick to raw observations and avoid making accusations that would tend to put someone on the defensive… and implying that someone else is committing violence (by communicating in a different style) is one of the most accusatory and putting-them-on-the-defensive things you can do. I’m curious, how many adherents of NVC are aware of this angle on it?
I don’t think NVC tries to put down an opponent, it’s mostly about how you present your ideas. I think it models an opponent as “he tries to win the debate without thinking about my goals. let me think of both mine and theirs goals, so i’m one step ahead”. Which is a bit prerogative and looking down, but not exactly accusatory