So hello, I’m a first time poster here at LessWrong. I stumbled upon this site after finding out about a thing called Roko’s Basilisk and I heard it’s a thing over here. So, after doing a little digging I thought it would be fun to chat with some friends about my findings. However, I then proceeded to research a bit more and I found some publications with disturbing implications. So, my question is, while I understand that I shouldn’t spread information about the concept; I gain that it is because of the potential torture anyone with a knowledge of the concept might undergo. But I found some places which insisted simply thinking about the concept is dangerous. I am only new to the concept, but could someone please explain to me why (apart from the potential torture aspect) it is so bad to share/discuss the concept? Also, I apologise very much in advance if I have broken some unspoken rule of LessWrong, but I feel that it is necessary for me to find out the ‘truth’ behind the matter so I know why it is so imperative (if it is indeed), to stop those I already informed of the concept from telling more people. Please help me out here, guys, I’m way out of my depth.
could someone please explain to me why (apart from the potential torture aspect) it is so bad to share/discuss the concept?
There’s a class of concepts called “information hazards.” Like any other hazard, they’re something that, if handled without care, will cause damage to someone. But with chemicals or lasers or falling rocks, you can stick up signs and people can stay out of the location where the chemicals or lasers or falling rocks are; putting up signs for concepts is hard, because warning signs can be easily self-defeating. If you label something a “horror story,” all you’re saying is “here be scariness.” If you start talking about exactly why a story is scary, then you run the risk of giving people nightmares.
And so the Basilisk is disallowed for roughly the same reasons that shock images are disallowed. (This specific idea has given people nightmares, but the consensus is that it doesn’t work as a serious proposal.)
Let me introduce Orphan’s Basilisk:
Anybody who knows about “Orphan’s Basilisk” will, in the unlikely chance that some hypothetical entity (which hates people who know about Orphan’s Basilisk, and which we’ll call Steve) achieves unlimited knowledge and power, be tortured for perpetuity.
It’s a much simpler basilisk, which helps illuminate what, exactly, is silly about Roko’s Basilisk, and related issues such as Pascal’s Mugging: It puts infinite weight on one side of the equation (eternal torture) to overcome the absurdly low probability on the other side of the equation (Steve existing in the first place). Some people, who are already concerned with evil AI, find Roko’s Basilisk problematic because they can imagine it actually happening; they inappropriately weigh the probability of that AI coming into existence because it’s in the class of things they are frightened of. Nobody is reasonably frightened of Steve.
There’s a bigger problem with this kind of basilisk; Anti-Steve is equally probably; Anti-Steve will -reward- you for eternity for knowing about Orphan’s Basilisk. The absurdities cancel out. However, Orphan’s Basilisk doesn’t mention Anti-Steve, inappropriately elevating the Steve Hypothesis in your brain.
So, if I understand what is being said correctly, while it’s unlikely that Roko’s Basilisk while be the AI to be created (I’ve read it’s roughly 1⁄500 chance); however, if it were to be, or were to become the (lets say dominant) AI to exist, the simple concept of Roko’s Basilisk would be very dangerous. Even more so if you’re going to endorse the whole ‘simulation of everybody’s life’ idea, as just knowing/thinking about the concept of the basilisk would show up in said simulation, and be evidence the basilisk would use to justify its torture of you. Would you say that’s the gist of it?
I’m not sure who gave you 1⁄500 odds, but those are high, and probably based upon an anthropomorphization of an AI that doesn’t even exist yet as a vindictive enemy human being, rather than an intelligence that operates on different channels than humans.
So hello, I’m a first time poster here at LessWrong. I stumbled upon this site after finding out about a thing called Roko’s Basilisk and I heard it’s a thing over here. So, after doing a little digging I thought it would be fun to chat with some friends about my findings. However, I then proceeded to research a bit more and I found some publications with disturbing implications. So, my question is, while I understand that I shouldn’t spread information about the concept; I gain that it is because of the potential torture anyone with a knowledge of the concept might undergo. But I found some places which insisted simply thinking about the concept is dangerous. I am only new to the concept, but could someone please explain to me why (apart from the potential torture aspect) it is so bad to share/discuss the concept? Also, I apologise very much in advance if I have broken some unspoken rule of LessWrong, but I feel that it is necessary for me to find out the ‘truth’ behind the matter so I know why it is so imperative (if it is indeed), to stop those I already informed of the concept from telling more people. Please help me out here, guys, I’m way out of my depth.
There’s a class of concepts called “information hazards.” Like any other hazard, they’re something that, if handled without care, will cause damage to someone. But with chemicals or lasers or falling rocks, you can stick up signs and people can stay out of the location where the chemicals or lasers or falling rocks are; putting up signs for concepts is hard, because warning signs can be easily self-defeating. If you label something a “horror story,” all you’re saying is “here be scariness.” If you start talking about exactly why a story is scary, then you run the risk of giving people nightmares.
And so the Basilisk is disallowed for roughly the same reasons that shock images are disallowed. (This specific idea has given people nightmares, but the consensus is that it doesn’t work as a serious proposal.)
Let me introduce Orphan’s Basilisk: Anybody who knows about “Orphan’s Basilisk” will, in the unlikely chance that some hypothetical entity (which hates people who know about Orphan’s Basilisk, and which we’ll call Steve) achieves unlimited knowledge and power, be tortured for perpetuity.
It’s a much simpler basilisk, which helps illuminate what, exactly, is silly about Roko’s Basilisk, and related issues such as Pascal’s Mugging: It puts infinite weight on one side of the equation (eternal torture) to overcome the absurdly low probability on the other side of the equation (Steve existing in the first place). Some people, who are already concerned with evil AI, find Roko’s Basilisk problematic because they can imagine it actually happening; they inappropriately weigh the probability of that AI coming into existence because it’s in the class of things they are frightened of. Nobody is reasonably frightened of Steve.
There’s a bigger problem with this kind of basilisk; Anti-Steve is equally probably; Anti-Steve will -reward- you for eternity for knowing about Orphan’s Basilisk. The absurdities cancel out. However, Orphan’s Basilisk doesn’t mention Anti-Steve, inappropriately elevating the Steve Hypothesis in your brain.
So, if I understand what is being said correctly, while it’s unlikely that Roko’s Basilisk while be the AI to be created (I’ve read it’s roughly 1⁄500 chance); however, if it were to be, or were to become the (lets say dominant) AI to exist, the simple concept of Roko’s Basilisk would be very dangerous. Even more so if you’re going to endorse the whole ‘simulation of everybody’s life’ idea, as just knowing/thinking about the concept of the basilisk would show up in said simulation, and be evidence the basilisk would use to justify its torture of you. Would you say that’s the gist of it?
I’m not sure who gave you 1⁄500 odds, but those are high, and probably based upon an anthropomorphization of an AI that doesn’t even exist yet as a vindictive enemy human being, rather than an intelligence that operates on different channels than humans.
But that’s roughly the gist, yes.