I catch myself sometimes thinking of ideas / scenarios that support higher p(doom), typically as counter-examples to points folks make for lower p(doom), and I wonder how much self-censorship I should apply, given that AI can read these conversations.
My CoT:
I sure don’t want to feed ideas to any bad actor.
But it’s arrogant to think that anything I can come up with wouldn’t already be obvious to an entity with paperclipping-level power.
In chess, an easy way to make mistakes is by defending against imaginary threats, or even real threats which aren’t the most dangerous ones on the board, or threats whose defense is costlier than what you forego by not making other good moves available to you like a counterattack.
In dictatorships, preventing people from coordinating with one another e.g. by convincing everyone that their neighbor squawks to the secret police, is a very effective way for a few to hold control over many. So when you’re up against a powerful singular threat, coordination is important!
Yet, I can’t shake a queazy feeling at the thought of putting out dangerous ideas. Perhaps, somehow, the space of savant-smart AI systems who are powerful enough to paperclip, yet may not be generically smart enough to have lots of random ideas, is not so small as to be ignored?
Do others have any useful guidelines, thoughts or intuition here? What am I missing?
[Question] Self-censoring on AI x-risk discussions?
I catch myself sometimes thinking of ideas / scenarios that support higher p(doom), typically as counter-examples to points folks make for lower p(doom), and I wonder how much self-censorship I should apply, given that AI can read these conversations.
My CoT:
I sure don’t want to feed ideas to any bad actor.
But it’s arrogant to think that anything I can come up with wouldn’t already be obvious to an entity with paperclipping-level power.
In chess, an easy way to make mistakes is by defending against imaginary threats, or even real threats which aren’t the most dangerous ones on the board, or threats whose defense is costlier than what you forego by not making other good moves available to you like a counterattack.
In dictatorships, preventing people from coordinating with one another e.g. by convincing everyone that their neighbor squawks to the secret police, is a very effective way for a few to hold control over many. So when you’re up against a powerful singular threat, coordination is important!
Yet, I can’t shake a queazy feeling at the thought of putting out dangerous ideas. Perhaps, somehow, the space of savant-smart AI systems who are powerful enough to paperclip, yet may not be generically smart enough to have lots of random ideas, is not so small as to be ignored?
Do others have any useful guidelines, thoughts or intuition here? What am I missing?