This is all very good reasoning, and you’re not the first to think of it, but for reasons that are obvious if you think some more about it it’s generally considered a REALLY bad idea to talk about it in public. PMing or emailing official SIAI people should get to link to safer avenues to discussing these kinds of basilisks.
it’s generally considered a REALLY bad idea to talk about it in public
Well, that’s kind of what the post is against. I think the outlook is quite positive, really; although there are a few arguments for general bad stuff (and really, when isn’t there a 10^-15 chance of horrible stuff), when it comes to actual decisions it’s irrational to be blackmailed in this way.
It would be more sensible to check with other people, rather than assuming it’s safe, before exposing the public to something that you know that a lot of people believe to be dangerous.
...before exposing the public to something that you know that a lot of people believe to be dangerous.
The pieces of the puzzle that Manfred put together can all be found on lesswrong. What do you suggest, that research into game and decision theory be banned?
You’re being facetious. No one is seriously disputing where the boundary between basilisk and non-basilisk lies, only what to do with the things on the basilisk side of the line.
No one is seriously disputing where the boundary between basilisk and non-basilisk lies...
This assumes that everyone knows where the boundary lies. The original post by Manfred either crossed the boundary or it didn’t. In the case that it didn’t, it only serves as a warning sign of where not to go. In the case that it did, how is your knowledge of the boundary not a case of hindsight bias?
PMing or emailing official SIAI people should get to link to safer avenues to discussing these kinds of basilisks.
Hmm, should I vote you up because what you’re saying is true, or should I vote you down because you are attracting attention to the parent post which harmful to think about?
If an idea is guessable, then it seems irrational to think it is harmful to communicate it to somebody, since they could have guessed it themselves. Given that this is a website about rationality, IMO we should be able to talk about the chain of reasoning that leads to the decision that this guessable idea is harmful to communicate, since there’s clearly a flaw in there somewhere.
Upvoted the parent because I think the harm here is imaginary. Absurdly large utilities do not describe non-absurdly-large brains, but they are not a surprising output from humans displaying fitness. (Hey, I know a large number! Look at me!)
These ideas have come up and were suppressed before, so this is not a specific criticism of the original post.
This is all very good reasoning, and you’re not the first to think of it, but for reasons that are obvious if you think some more about it it’s generally considered a REALLY bad idea to talk about it in public. PMing or emailing official SIAI people should get to link to safer avenues to discussing these kinds of basilisks.
Well, that’s kind of what the post is against. I think the outlook is quite positive, really; although there are a few arguments for general bad stuff (and really, when isn’t there a 10^-15 chance of horrible stuff), when it comes to actual decisions it’s irrational to be blackmailed in this way.
It would be more sensible to check with other people, rather than assuming it’s safe, before exposing the public to something that you know that a lot of people believe to be dangerous.
The pieces of the puzzle that Manfred put together can all be found on lesswrong. What do you suggest, that research into game and decision theory be banned?
You’re being facetious. No one is seriously disputing where the boundary between basilisk and non-basilisk lies, only what to do with the things on the basilisk side of the line.
This assumes that everyone knows where the boundary lies. The original post by Manfred either crossed the boundary or it didn’t. In the case that it didn’t, it only serves as a warning sign of where not to go. In the case that it did, how is your knowledge of the boundary not a case of hindsight bias?
(The voters seem to think I’m being stupid, but that doesn’t actually tell me what the right answer is...)
First, I’m assuming that, in general, people who have not seen the basilisk are not going to mention it accidentally.
Second, I’m assuming that, due to the nature of the basilisk, those who have seen it know what is and is not basilisk-information.
Which of these two assumptions do you disagree with? (please check all that apply)
Hmm, should I vote you up because what you’re saying is true, or should I vote you down because you are attracting attention to the parent post which harmful to think about?
If an idea is guessable, then it seems irrational to think it is harmful to communicate it to somebody, since they could have guessed it themselves. Given that this is a website about rationality, IMO we should be able to talk about the chain of reasoning that leads to the decision that this guessable idea is harmful to communicate, since there’s clearly a flaw in there somewhere.
Upvoted the parent because I think the harm here is imaginary. Absurdly large utilities do not describe non-absurdly-large brains, but they are not a surprising output from humans displaying fitness. (Hey, I know a large number! Look at me!)
These ideas have come up and were suppressed before, so this is not a specific criticism of the original post.