I’m about to process the last few days worth of posts and comments. I’ll be linking to this comment as a “here are my current guesses for how to handle various moderation calls”.
When we’re processing many new user-posts a day, we don’t have much time to evaluate each post.
So, one principle I think is fairly likely to become a “new user guideline” is “Make it pretty clear off the bat what the point of the post is.” In ~3 sentences, try to make it clear who your target audience is, and what core point you’re trying to communicate to them. If you’re able to quickly gesture at the biggest-bit-of-evidence or argument that motivates your point, even better. (Though I understand sometimes this is hard).
This isn’t necessarily how you have to write all the time on LessWrong! But your first post is something like an admissions-essay and should be optimized more for being legibly coherent and useful. (And honestly I think most LW posts should lean more in this direction)
In some sense this is similar to submitting something to a journal or magazine. Editors get tons of submissions. For your first couple posts, don’t aim to write something that takes a lot of works for us to evaluate.
Corollary: Posts that are more likely to end up in the reject pile include...
Fiction, especially if it looks like it’s trying to make some kind of philosophical point, while being wrapped in a structure that makes that harder to evaluate. (I think good fiction plays a valuable role on LessWrong, I just don’t recommend it until you’ve gotten more of a handle of the culture and background knowledge)
Long manifestos. Like fiction, these are sometimes valuable. They can communicate something like an overarching way-of-seeing-the-world that is valuable in a different way from individual factual claims. But, a) I think it’s a reasonable system to first make some more succinct posts, build up some credibility, and then ask the LessWrongOSphere to evaluate your lengthy treatise. b) honestly… your first manifesto just probably isn’t very good. That’s okay. No judgment. I’ve written manifestos that weren’t very good and they were an important part of my learning process. Even my more recent manifestos tend to be less well received than my posts that argue a particular object-level claim.
Some users have asked: “Okay, but, when will I be allowed to post the long poetic prose that expresses the nuances of the idea I have in my heart?”
Often the answer is, well, when you get better at thinking and expressing yourself clearly enough that you’ve written a significantly different piece.
I’ve always like the the pithy advice “you have to know the rules to break the rules” which I do consider valid in many domains.
Before I let users break generally good rules like “explain what your point it up front”, I want to know that they could keep to the rule before they don’t. The posts of many first time users give me the feeling that their author isn’t being rambly on purpose, they don’t know how to write otherwise (or aren’t willing to).
Some top-level post topics that get much higher scrutiny:
1. Takes on AI
Simply because of the volume of it, the standards are higher. I recommend reading Scott Alexander’s Superintelligence FAQ as a good primer to make sure you understand the basics. Make sure you’re familiar with the Orthogonality Thesis and Instrumental Convergence. I recommend both Eliezer’s AGI Ruin: A List of Lethalities and Paul Christiano’s response post so you understand what sort of difficulties the field is actually facing.
2. Quantum Suicide/Immortality, Roko’s Basilisk and Acausal Extortion.
In theory, these are topics that have room for novel questions and contributions. In practice, they seem to attract people who seem… looking for something to be anxious about? I don’t have great advice for these people, but my impression is that they’re almost always trapped in a loop where they’re trying to think about it in enough detail that they don’t have to be anxious anymore, but that doesn’t work. They just keep finding new subthreads to be anxious about.
For Acausal Trade, I do think Critch’s Acausal normalcy might be a useful perspective that points your thoughts in a more useful direction. Alas, I don’t have a great primer that succinctly explains why quantum immortality isn’t a great frame, in a way that doesn’t have a ton of philosophical dependencies.
I mostly recommend… going outside, hanging out with friends and finding other more productive things to get intellectually engrossed in.
3. Needing help with depression, akrasia, or medical advice with confusing mystery illness.
This is pretty sad and I feel quite bad saying it – on one hand, I do think LessWrong has some useful stuff to offer here. But too much focus on this has previously warped the community in weird ways – people with all kinds of problems come trying to get help and we just don’t have the resources to help all of them.
For your first post on LessWrong, think of it more like you’re applying to a university. Yes, universities have mental health departments for students and faculty… but when we’re evaluating “does it make sense to let this person into this university”, the focus should be on “does this person have the ability to make useful intellectual contributions?” not “do they need help in a way we can help with?”
3. Needing help with depression, akrasia, or medical advice with confusing mystery illness.
Bit of a shame to see this one, but I understand this one. It’s crunch time for AGI alignment and there’s a lot on the line. Maybe those of us interested in self-help can go to/post their thoughts on some of the rationalsphere blogs, or maybe start their own.
I got a lot of value out of the more self-help and theory of mind posts here, especially Kaj Sotala’s and Valentine’s work on multiagent models of mind, and it’d be cool to have another place to continue discussions around that.
A key question when I look at a new user on LessWrong trying to help with AI is, well, are they actually likely to be able to contribute to the field of AI safety?
If they are aiming to make direct novel intellectual contributions, this is in fact fairly hard. People have argued back and forth about how much raw IQ, conscientiousness or other signs of promise a person needs to have. There has been some posts arguing that people are overly pessimistic and gatekeeping-y about AI safety.
But, I think it’s just pretty importantly true that it takes a fairly significant combination of intelligence and dedication to contribute. Not everyone is cut out for doing original research. Many people pre-emptively focus on community building and governance because that feels easier and more tractable to them than original research. But those areas still require you to have a pretty understanding of the field you’re trying to govern or build a community for.
If someone writes a post on AI that seems like a bad take, which isn’t really informed by the real challenges, should I be encouraging that person to make improvements and try again? Or just say “idk man, not everyone is cut out for this?”
Here’s my current answer.
If you’ve written a take on AI that didn’t seem to hit the LW team’s quality bar, I would recommend some combination of:
Read ~16 hours of background content, so you’re not just completely missing the point. (I have some material in mind that I’ll compile later, but for now highlight roughly the amount of effort involved)
Set aside ~4 hours to think seriously about the topic. Try to find one sub-question you don’t know the answer to, and make progress answering that sub-question.
Write up your thoughts as a LW post.
(For each of these steps, organizing some friends to work together as a reading or thinking group can be helpful to make it more fun)
This doesn’t guarantee that you’ll be a good fit for AI safety work, but I think this is an amount of effort where it’s possible for a LW mod to look at your work, and figure out if this is likely to be a good use of your time.
Some people may object “this is a lot of work.” Yes, it is. If you’re the right sort of person you may just find this work fun. But the bottom line is yes, this is work. You should not expect to contribute to the field without putting in serious work, and I’m basically happy to filter out of LessWrong people who a) seem to superficially have pretty confused takes, and b) are unwilling to put in 20 hours of research work.
Draft in progress. Common failures modes for AI posts that I want to reference later:
Trying to help with AI Alignment
“Let’s make the AI not do anything.”
This is essentially a very expensive rock. Other people will be building AIs that do do stuff. How does your AI help the situation over not building anything at all?
“Let’s make the AI do [some specific thing that seems maybe helpful when parsed as an english sentence], without actually describing how to make sure they do exactly or even approximately that english sentence”
The problem is a) we don’t know how to point an AI at doing anything at all, and b) your simple english sentence includes a ton of hidden assumptions.
(Note: I think Mark Xu sort of disagreed with Oli on something related to this recently, so I don’t know that I consider this class of solution is completely settled. I think Mark Xu thinks that we don’t currently know how to get an AI to do moderately complicated actions with our current tech, but, our current paradigms for how to train AIs are likely to yield AIs that can do moderately complicated actions)
I think the typical new user who says things like this still isn’t advancing the current paradigm though, nor saying anything useful that hasn’t already been said.
Arguing Alignment is Doomed
[less well formulated]
Lately there’s been a crop of posts arguing alignment is doomed. I… don’t even strongly disagree with them, but they tend to be poorly argued and seem confused about what good problem solving looks like.
Arguing AI Risk is a dumb concern that doesn’t make sense
Lately (in particular since Eliezer’s TIME article), we’ve had a bunch of people coming in to say we’re a bunch of doomsday cultists and/or gish gallopers.
And, well, I think if you’re just tuning into the TIME article, or you have only been paying bits of attention over the years, I think this is a kinda reasonable belief-state to have. From the outside when you hear an extreme-sounding claim, it’s reasonable for alarm bells to go off and assume this is maybe crazy.
If you were the first person bringing up this concern, I’d be interested in your take. But, we’ve had a ton of these, so you’re not saying anything new by bringing it up.
You’re welcome to post your take somewhere else, but if you want to participate on LessWrong, you need to engage with the object level arguments.
Here’s a couple things I’ll say about this:
One particularly gishgallopy-feeling thing is that many arguments for AI catastrophe are disjunctive. So, yeah, there’s not just one argument you can overturn and then we’ll all be like “okay great, we can change our mind about this problem.” BUT, it is the case that we are pretty curious about individual arguments getting overturned. If individual disjunctive arguments turned out to be flawed, that’d make the problem easier. So I’d be fairly excited about someone who digs into the details of various claims in AGI Ruin: A List of Lethalities and either disproves them or finds a way around them.
Another potentially gishgallopy-feeling thing is If you’re discussing things on LessWrong, you’ll be expected to have absorbed the concepts from the sequences (such as how to think about subjective probability, tribalism, etc), either by reading the sequences or lurking a lot. I acknowledge this is as pretty gishgallopy at first glance, if you came here to debate one particular thing. Alas, that’s just how it is [see other FAQ question delving more into this]
Here’s a quickly written draft for an FAQ we might send users whose content gets blocked from appearing on the site.
The “My post/comment was rejected” FAQ
Why was my submission rejected?
Common reasons that the LW Mod team will reject your post or comment:
It fails to acknowledge or build upon standard responses and objections that are well-known on LessWrong.
The LessWrong website is 14 years old and the community behind it older still. Our core readings [link] are over half a million words. So understandably, there’s a lot you might have missed!
Unfortunately, as the amount of interest in LessWrong grows, we can’t afford to let cutting-edge content get submerged under content from people who aren’t yet caught up to the rest of the site.
It is poorly reasoned. It contains some mix of bad arguments and obviously bad positions that it does not feel worth the LessWrong mod team or LessWrong community’s time or effort responding to.
It is difficult to read. Not all posts and comments are equally well-written and make their points as clearly. While established users might get more charity, for new users, we require that we (and others) can easily follow what you’re saying well or enough to know whether or not you’re saying something interesting.
There are multiple ways to end up difficult to read:
Poorly crafted sentences and paragraphs
Long and rambly
Poorly structure without signposting
Poor formatting
It’s a post and doesn’t say very much.
Each post takes up some space and requires effort to click on and read, and the content of the post needs to make that worthwhile. Your post might have had a reasonable thought, but it didn’t justify a top-level post.
If it’s a quick thought about AI, try an “AI No Dumb Questions Open Thread”
Sometimes this just means you need to put in more effort, though effort isn’t the core thing.
You are rude or hostile. C’mon.
Can I appeal or try again?
You are welcome to message us however note that due to volume, even though we read most messages, we will not necessarily respond, and if we do, we can’t engage in a lengthy back-and-forth.
In an ideal world, we’d have a lot more capacity to engage with each new contributor to discuss what was/wasn’t good about their content, unfortunately with new submissions increasing on short timescales, we can’t afford that and have to be pretty strict in order to ensure site quality stays high.
This is censorship, etc.
LessWrong, while on the public internet, is not the general public. It was built to host a certain kind of discussion between certain kinds of users who’ve agreed to certain basics of discourse, who are on board with a shared philosophy, and can assume certain background knowledge.
Sometimes we say that LessWrong is a little bit like a “university”, and one way that it is true is that not anybody is entitled to walk in and demand that people host the conversation they like.
We hope to keep allowing for new accounts created and for new users to submit new content, but the only way we can do that is if we reject content and users who would degrade the site’s standards.
But diversity of opinion is important, echo chamber, etc.
That’s a good point and a risk that we face when moderating. The LessWrong moderation team tries hard to not reject things just because we disagree, and instead only do so if it feels like the content it failing on some other criteria.
Notably, one of the best ways to disagree (and that will likely get you upvotes) is to criticize not just a commonly-held position on LessWrong, but the reasons why it is held. If you show that you understand why people believe what they do, they’re much more likely to be interested in your criticisms.
How is the mod team kept accountable?
If your post or comment was rejected from the main site, it will be viewable alongside some indication for why it was banned. [we haven’t built this yet but plan to soon]
Anyone who wants to audit our decision-making and moderation policies can review blocked content there.
It fails to acknowledge or build upon standard responses and objections that are well-known on LessWrong.
The LessWrong website is 14 years old and the community behind it older still. Our core readings [link] are over half a million words. So understandably, there’s a lot you might have missed!
Unfortunately, as the amount of interest in LessWrong grows, we can’t afford to let cutting-edge content get submerged under content from people who aren’t yet caught up to the rest of the site.
I do want to emphasize a subtle distinction between “you have brought up arguments that have already been brought up” and “you are challenging basic assumptions of the ideas here”. I think challenging basic assumptions is good and well (like here), while bringing up “but general intelligences can’t exist because of no-free-lunch theorems” or “how could a computer ever do any harm, we can just unplug it” is quite understandably met with “we’ve spent 100s or 1000s of hours discussing and rebutting that specific argument, please go read about it <here> and come back when you’re confident you’re not making the same arguments as the last dozen new users”.
I would like to make sure new users are specifically not given the impression that LW mods aren’t open to basic assumptions being challenged, and I think it might be worth the space to specifically rule out that interpretation somehow.
Well, I don’t think you have to acknowledge existing stuff on this topic if you have a new and good argument.
Added: I think the phrasing I’d prefer is “You made an argument that has already been addressed extensively on LessWrong” rather than “You have talked about a topic without reading everything we’ve already written about on that topic”.
I do think there is an interesting question of “how much should people have read?” which is actually hard to answer.
There are people who don’t need to read as much in order to say sensible and valuable things, and some people that no amount of reading seems to save.
The half a million words is the Sequences. I don’t obviously want a rule that says you need to have read all of them in order to post/comment (nor do I think doing so is a guarantee), but also I do want to say that if you make mistakes the the Sequences would teach you not to make, that could be grounds for not having your content approved.
A lot of the AI newbie stuff I’m disinclined to approve is the kind that makes claims that are actually countered in the Sequences, e.g. orthogonality thesis, treating the AI too much like humans, various fallacies involving words.
How do you know you have a new and good argument if you don’t know the standards thing said on the topic
And relatedly, why should I or other readers on LW assume that you have a new and good argument without any indication that you know the arguments in general?
This is aimed at users making their very first post/comment. I think it is likely a good policy/heuristic for the mod team in judging your post that claims “AIs won’t be dangerous because X”, tells me early on that you’re not wasting my time because you’re already aware of all the standard arguments.
In a world where everyday a few dozen people who started thinking about AI two weeks ago show up on LessWrong and want to give their “why not just X?”, I think it’s reasonable to say “we want you to give some indication that you’re aware of the basic discussion this site generally assumes”.
I find it hilarious that you can say this, while simultaneously, the vast majority of this community is deeply upset they are being ignored by academia and companies, because they often have no formal degrees or peer reviewed publication, or other evidence of having considered the relevant science. Less wrong fails the standards of these fields and areas. You routinely re-invent concepts that already exist. Or propose solutions that would be immediately rejected as infeasible if you tried to get them into a journal.
Explaining a concept your community takes for granted to outsiders can help you refresh it, understand it better, and spot potential problems. A lot of things taken for granted here are rejected by outsiders because they are not objectively plausible.
And a significant number of newcomers, while lacking LW canon, will have other relevant knowledge. If you make the bar too high, you deter them.
All this is particularly troubling because your canon is spread all over the place, extremely lengthy, and individually usually incomplete or outdated, and filled in implicitly from prior forum interactions. Academic knowledge is more accessible that way.
Writing hastily in the interests of time, sorry if not maximally clear.
Explaining a concept your community takes for granted to outsiders can help you refresh it, understand it better, and spot potential problems.
It’s very much a matter of how many newcomers there are relative to existing members. If the number of existing members is large compared to newcomers, it’s not so bad to take the time to explain things.
If the number of newcomers threatens to overwhelm the existing community, it’s just not practical to let everyone in. Among other factors, certain conversation is possible because you can assume that most people have certain background and even if they disagree, at least know the things you know.
The need for getting stricter is because of the current (and forecasted) increase in new user. This means we can’t afford to become 50% posts that ignore everything our community has already figured out.
LessWrong is an internet forum, but it’s in the direction of a university/academic publication, and such publications only work because editors don’t accept everything.
the vast majority of this community is deeply upset they are being ignored by academia and companies, because they often have no formal degrees or peer reviewed publication, or other evidence of having considered the relevant science.
my guess is that that claim is slightly exaggerated, but I expect sources exist for an only mildly weaker claim. I certainly have been specifically mocked for my username in places that watch this site, for example.
I certainly have been specifically mocked for my username in places that watch this site, for example.
This is an example of people mocking LW for something. Portia is making a claim about LW users’ internal emotional states; she is asserting that they care deeply about academic recognition and feel infuriated they’re not getting it. Does this describe you or the rest of the website, in your experience?
lens portia’s writing out of frustrated tone first and it makes more sense. they’re saying that recognition is something folks care about (yeah, I think so) and aren’t getting to an appropriate degree (also seems true). like I said in my other comment—tone makes it harder to extract intended meaning.
Well, I disagree. I have literally zero interest in currying the favor of academics, and think Portia is projecting a respect and yearning for status within universities onto the rest of us that mostly doesn’t exist. I would additionally prefer if this community were able to set standards for its members without having to worry about or debate whether or not asking people to read canon is a status grab.
sure. I do think it’s helpful to be academically valid sometimes though. you don’t need to care, but some do some of the time somewhat. maybe not as much as the literal wording used here. catch ya later, anyhow.
strong agree, single upvote: harsh tone, but reasonable message. I hope the tone doesn’t lead this point to be ignored, as I do think it’s important. but leading with mocking does seem like it’s probably why others have downvoted. downvote need not indicate refusal to consider, merely negative feedback to tone, but I worry about that, given the agree votes are also in the negative.
Thank you. And I apologise for the tone. I think the back of my mind was haunted by Shoggoth with a Smiley face giving me advice for my weekend plans, and that emotional turmoil came out the wrong way.
I am in the strange position of being on this forum, and in academia, and seeing both sides engage in the same barrier keeping behaviour, and call it out as elitist and misguided in the other but a necessary way to ensure quality and affirm your superior identity in your own group is jarring. I’ve found valuable and admirable practices and insights in both, else I would not be there.
Any group that bears a credential, or performs negative selection of some kind, will bear the traits you speak of. ’Tis the nature of most task-performing groups human society produces. Alas, one cannot escape it, even coming to a group that once claimed to eschew credentialism. Nonetheless, it is still worthwhile to engage with these groups intellectually.
We didn’t build that yet but plan to soon. (I think what happened was Ruby wrote this up in a private google doc, I encouraged him to post it as a comment so I could link to it, and both of us forgot it included that explicit link. Sorry about that, I’ll edit it to clarify)
I’m about to process the last few days worth of posts and comments. I’ll be linking to this comment as a “here are my current guesses for how to handle various moderation calls”.
Succinctly explain the main point.
When we’re processing many new user-posts a day, we don’t have much time to evaluate each post.
So, one principle I think is fairly likely to become a “new user guideline” is “Make it pretty clear off the bat what the point of the post is.” In ~3 sentences, try to make it clear who your target audience is, and what core point you’re trying to communicate to them. If you’re able to quickly gesture at the biggest-bit-of-evidence or argument that motivates your point, even better. (Though I understand sometimes this is hard).
This isn’t necessarily how you have to write all the time on LessWrong! But your first post is something like an admissions-essay and should be optimized more for being legibly coherent and useful. (And honestly I think most LW posts should lean more in this direction)
In some sense this is similar to submitting something to a journal or magazine. Editors get tons of submissions. For your first couple posts, don’t aim to write something that takes a lot of works for us to evaluate.
Corollary: Posts that are more likely to end up in the reject pile include...
Fiction, especially if it looks like it’s trying to make some kind of philosophical point, while being wrapped in a structure that makes that harder to evaluate. (I think good fiction plays a valuable role on LessWrong, I just don’t recommend it until you’ve gotten more of a handle of the culture and background knowledge)
Long manifestos. Like fiction, these are sometimes valuable. They can communicate something like an overarching way-of-seeing-the-world that is valuable in a different way from individual factual claims. But, a) I think it’s a reasonable system to first make some more succinct posts, build up some credibility, and then ask the LessWrongOSphere to evaluate your lengthy treatise. b) honestly… your first manifesto just probably isn’t very good. That’s okay. No judgment. I’ve written manifestos that weren’t very good and they were an important part of my learning process. Even my more recent manifestos tend to be less well received than my posts that argue a particular object-level claim.
Some users have asked: “Okay, but, when will I be allowed to post the long poetic prose that expresses the nuances of the idea I have in my heart?”
Often the answer is, well, when you get better at thinking and expressing yourself clearly enough that you’ve written a significantly different piece.
I’ve always like the the pithy advice “you have to know the rules to break the rules” which I do consider valid in many domains.
Before I let users break generally good rules like “explain what your point it up front”, I want to know that they could keep to the rule before they don’t. The posts of many first time users give me the feeling that their author isn’t being rambly on purpose, they don’t know how to write otherwise (or aren’t willing to).
Some top-level post topics that get much higher scrutiny:
1. Takes on AI
Simply because of the volume of it, the standards are higher. I recommend reading Scott Alexander’s Superintelligence FAQ as a good primer to make sure you understand the basics. Make sure you’re familiar with the Orthogonality Thesis and Instrumental Convergence. I recommend both Eliezer’s AGI Ruin: A List of Lethalities and Paul Christiano’s response post so you understand what sort of difficulties the field is actually facing.
I suggest going to the most recent AI Open Questions thread, or looking into the FAQ at https://ui.stampy.ai/
2. Quantum Suicide/Immortality, Roko’s Basilisk and Acausal Extortion.
In theory, these are topics that have room for novel questions and contributions. In practice, they seem to attract people who seem… looking for something to be anxious about? I don’t have great advice for these people, but my impression is that they’re almost always trapped in a loop where they’re trying to think about it in enough detail that they don’t have to be anxious anymore, but that doesn’t work. They just keep finding new subthreads to be anxious about.
For Acausal Trade, I do think Critch’s Acausal normalcy might be a useful perspective that points your thoughts in a more useful direction. Alas, I don’t have a great primer that succinctly explains why quantum immortality isn’t a great frame, in a way that doesn’t have a ton of philosophical dependencies.
I mostly recommend… going outside, hanging out with friends and finding other more productive things to get intellectually engrossed in.
3. Needing help with depression, akrasia, or medical advice with confusing mystery illness.
This is pretty sad and I feel quite bad saying it – on one hand, I do think LessWrong has some useful stuff to offer here. But too much focus on this has previously warped the community in weird ways – people with all kinds of problems come trying to get help and we just don’t have the resources to help all of them.
For your first post on LessWrong, think of it more like you’re applying to a university. Yes, universities have mental health departments for students and faculty… but when we’re evaluating “does it make sense to let this person into this university”, the focus should be on “does this person have the ability to make useful intellectual contributions?” not “do they need help in a way we can help with?”
Maybe an FAQ for the intersection of #1, #2 and #3, “depressed/anxious because of AI”, might be a good thing to be able to link to, though?
Bit of a shame to see this one, but I understand this one. It’s crunch time for AGI alignment and there’s a lot on the line. Maybe those of us interested in self-help can go to/post their thoughts on some of the rationalsphere blogs, or maybe start their own.
I got a lot of value out of the more self-help and theory of mind posts here, especially Kaj Sotala’s and Valentine’s work on multiagent models of mind, and it’d be cool to have another place to continue discussions around that.
A key question when I look at a new user on LessWrong trying to help with AI is, well, are they actually likely to be able to contribute to the field of AI safety?
If they are aiming to make direct novel intellectual contributions, this is in fact fairly hard. People have argued back and forth about how much raw IQ, conscientiousness or other signs of promise a person needs to have. There has been some posts arguing that people are overly pessimistic and gatekeeping-y about AI safety.
But, I think it’s just pretty importantly true that it takes a fairly significant combination of intelligence and dedication to contribute. Not everyone is cut out for doing original research. Many people pre-emptively focus on community building and governance because that feels easier and more tractable to them than original research. But those areas still require you to have a pretty understanding of the field you’re trying to govern or build a community for.
If someone writes a post on AI that seems like a bad take, which isn’t really informed by the real challenges, should I be encouraging that person to make improvements and try again? Or just say “idk man, not everyone is cut out for this?”
Here’s my current answer.
If you’ve written a take on AI that didn’t seem to hit the LW team’s quality bar, I would recommend some combination of:
Read ~16 hours of background content, so you’re not just completely missing the point. (I have some material in mind that I’ll compile later, but for now highlight roughly the amount of effort involved)
Set aside ~4 hours to think seriously about the topic. Try to find one sub-question you don’t know the answer to, and make progress answering that sub-question.
Write up your thoughts as a LW post.
(For each of these steps, organizing some friends to work together as a reading or thinking group can be helpful to make it more fun)
This doesn’t guarantee that you’ll be a good fit for AI safety work, but I think this is an amount of effort where it’s possible for a LW mod to look at your work, and figure out if this is likely to be a good use of your time.
Some people may object “this is a lot of work.” Yes, it is. If you’re the right sort of person you may just find this work fun. But the bottom line is yes, this is work. You should not expect to contribute to the field without putting in serious work, and I’m basically happy to filter out of LessWrong people who a) seem to superficially have pretty confused takes, and b) are unwilling to put in 20 hours of research work.
Draft in progress. Common failures modes for AI posts that I want to reference later:
Trying to help with AI Alignment
“Let’s make the AI not do anything.”
This is essentially a very expensive rock. Other people will be building AIs that do do stuff. How does your AI help the situation over not building anything at all?
“Let’s make the AI do [some specific thing that seems maybe helpful when parsed as an english sentence], without actually describing how to make sure they do exactly or even approximately that english sentence”
The problem is a) we don’t know how to point an AI at doing anything at all, and b) your simple english sentence includes a ton of hidden assumptions.
(Note: I think Mark Xu sort of disagreed with Oli on something related to this recently, so I don’t know that I consider this class of solution is completely settled. I think Mark Xu thinks that we don’t currently know how to get an AI to do moderately complicated actions with our current tech, but, our current paradigms for how to train AIs are likely to yield AIs that can do moderately complicated actions)
I think the typical new user who says things like this still isn’t advancing the current paradigm though, nor saying anything useful that hasn’t already been said.
Arguing Alignment is Doomed
[less well formulated]
Lately there’s been a crop of posts arguing alignment is doomed. I… don’t even strongly disagree with them, but they tend to be poorly argued and seem confused about what good problem solving looks like.
Arguing AI Risk is a dumb concern that doesn’t make sense
Lately (in particular since Eliezer’s TIME article), we’ve had a bunch of people coming in to say we’re a bunch of doomsday cultists and/or gish gallopers.
And, well, I think if you’re just tuning into the TIME article, or you have only been paying bits of attention over the years, I think this is a kinda reasonable belief-state to have. From the outside when you hear an extreme-sounding claim, it’s reasonable for alarm bells to go off and assume this is maybe crazy.
If you were the first person bringing up this concern, I’d be interested in your take. But, we’ve had a ton of these, so you’re not saying anything new by bringing it up.
You’re welcome to post your take somewhere else, but if you want to participate on LessWrong, you need to engage with the object level arguments.
Here’s a couple things I’ll say about this:
One particularly gishgallopy-feeling thing is that many arguments for AI catastrophe are disjunctive. So, yeah, there’s not just one argument you can overturn and then we’ll all be like “okay great, we can change our mind about this problem.” BUT, it is the case that we are pretty curious about individual arguments getting overturned. If individual disjunctive arguments turned out to be flawed, that’d make the problem easier. So I’d be fairly excited about someone who digs into the details of various claims in AGI Ruin: A List of Lethalities and either disproves them or finds a way around them.
Another potentially gishgallopy-feeling thing is If you’re discussing things on LessWrong, you’ll be expected to have absorbed the concepts from the sequences (such as how to think about subjective probability, tribalism, etc), either by reading the sequences or lurking a lot. I acknowledge this is as pretty gishgallopy at first glance, if you came here to debate one particular thing. Alas, that’s just how it is [see other FAQ question delving more into this]
Here’s a quickly written draft for an FAQ we might send users whose content gets blocked from appearing on the site.
The “My post/comment was rejected” FAQ
Why was my submission rejected?
Common reasons that the LW Mod team will reject your post or comment:
It fails to acknowledge or build upon standard responses and objections that are well-known on LessWrong.
The LessWrong website is 14 years old and the community behind it older still. Our core readings [link] are over half a million words. So understandably, there’s a lot you might have missed!
Unfortunately, as the amount of interest in LessWrong grows, we can’t afford to let cutting-edge content get submerged under content from people who aren’t yet caught up to the rest of the site.
It is poorly reasoned. It contains some mix of bad arguments and obviously bad positions that it does not feel worth the LessWrong mod team or LessWrong community’s time or effort responding to.
It is difficult to read. Not all posts and comments are equally well-written and make their points as clearly. While established users might get more charity, for new users, we require that we (and others) can easily follow what you’re saying well or enough to know whether or not you’re saying something interesting.
There are multiple ways to end up difficult to read:
Poorly crafted sentences and paragraphs
Long and rambly
Poorly structure without signposting
Poor formatting
It’s a post and doesn’t say very much.
Each post takes up some space and requires effort to click on and read, and the content of the post needs to make that worthwhile. Your post might have had a reasonable thought, but it didn’t justify a top-level post.
If it’s a quick thought about AI, try an “AI No Dumb Questions Open Thread”
Sometimes this just means you need to put in more effort, though effort isn’t the core thing.
You are rude or hostile. C’mon.
Can I appeal or try again?
You are welcome to message us however note that due to volume, even though we read most messages, we will not necessarily respond, and if we do, we can’t engage in a lengthy back-and-forth.
In an ideal world, we’d have a lot more capacity to engage with each new contributor to discuss what was/wasn’t good about their content, unfortunately with new submissions increasing on short timescales, we can’t afford that and have to be pretty strict in order to ensure site quality stays high.
This is censorship, etc.
LessWrong, while on the public internet, is not the general public. It was built to host a certain kind of discussion between certain kinds of users who’ve agreed to certain basics of discourse, who are on board with a shared philosophy, and can assume certain background knowledge.
Sometimes we say that LessWrong is a little bit like a “university”, and one way that it is true is that not anybody is entitled to walk in and demand that people host the conversation they like.
We hope to keep allowing for new accounts created and for new users to submit new content, but the only way we can do that is if we reject content and users who would degrade the site’s standards.
But diversity of opinion is important, echo chamber, etc.
That’s a good point and a risk that we face when moderating. The LessWrong moderation team tries hard to not reject things just because we disagree, and instead only do so if it feels like the content it failing on some other criteria.
Notably, one of the best ways to disagree (and that will likely get you upvotes) is to criticize not just a commonly-held position on LessWrong, but the reasons why it is held. If you show that you understand why people believe what they do, they’re much more likely to be interested in your criticisms.
How is the mod team kept accountable?
If your post or comment was rejected from the main site, it will be viewable alongside some indication for why it was banned. [we haven’t built this yet but plan to soon]
Anyone who wants to audit our decision-making and moderation policies can review blocked content there.
I do want to emphasize a subtle distinction between “you have brought up arguments that have already been brought up” and “you are challenging basic assumptions of the ideas here”. I think challenging basic assumptions is good and well (like here), while bringing up “but general intelligences can’t exist because of no-free-lunch theorems” or “how could a computer ever do any harm, we can just unplug it” is quite understandably met with “we’ve spent 100s or 1000s of hours discussing and rebutting that specific argument, please go read about it <here> and come back when you’re confident you’re not making the same arguments as the last dozen new users”.
I would like to make sure new users are specifically not given the impression that LW mods aren’t open to basic assumptions being challenged, and I think it might be worth the space to specifically rule out that interpretation somehow.
Can be made more explicit, but this is exactly why the section opens with “acknowledge [existing stuff on topic]”.
Well, I don’t think you have to acknowledge existing stuff on this topic if you have a new and good argument.
Added: I think the phrasing I’d prefer is “You made an argument that has already been addressed extensively on LessWrong” rather than “You have talked about a topic without reading everything we’ve already written about on that topic”.
I do think there is an interesting question of “how much should people have read?” which is actually hard to answer.
There are people who don’t need to read as much in order to say sensible and valuable things, and some people that no amount of reading seems to save.
The half a million words is the Sequences. I don’t obviously want a rule that says you need to have read all of them in order to post/comment (nor do I think doing so is a guarantee), but also I do want to say that if you make mistakes the the Sequences would teach you not to make, that could be grounds for not having your content approved.
A lot of the AI newbie stuff I’m disinclined to approve is the kind that makes claims that are actually countered in the Sequences, e.g. orthogonality thesis, treating the AI too much like humans, various fallacies involving words.
How do you know you have a new and good argument if you don’t know the standards thing said on the topic
And relatedly, why should I or other readers on LW assume that you have a new and good argument without any indication that you know the arguments in general?
This is aimed at users making their very first post/comment. I think it is likely a good policy/heuristic for the mod team in judging your post that claims “AIs won’t be dangerous because X”, tells me early on that you’re not wasting my time because you’re already aware of all the standard arguments.
In a world where everyday a few dozen people who started thinking about AI two weeks ago show up on LessWrong and want to give their “why not just X?”, I think it’s reasonable to say “we want you to give some indication that you’re aware of the basic discussion this site generally assumes”.
I find it hilarious that you can say this, while simultaneously, the vast majority of this community is deeply upset they are being ignored by academia and companies, because they often have no formal degrees or peer reviewed publication, or other evidence of having considered the relevant science. Less wrong fails the standards of these fields and areas. You routinely re-invent concepts that already exist. Or propose solutions that would be immediately rejected as infeasible if you tried to get them into a journal.
Explaining a concept your community takes for granted to outsiders can help you refresh it, understand it better, and spot potential problems. A lot of things taken for granted here are rejected by outsiders because they are not objectively plausible.
And a significant number of newcomers, while lacking LW canon, will have other relevant knowledge. If you make the bar too high, you deter them.
All this is particularly troubling because your canon is spread all over the place, extremely lengthy, and individually usually incomplete or outdated, and filled in implicitly from prior forum interactions. Academic knowledge is more accessible that way.
Writing hastily in the interests of time, sorry if not maximally clear.
It’s very much a matter of how many newcomers there are relative to existing members. If the number of existing members is large compared to newcomers, it’s not so bad to take the time to explain things.
If the number of newcomers threatens to overwhelm the existing community, it’s just not practical to let everyone in. Among other factors, certain conversation is possible because you can assume that most people have certain background and even if they disagree, at least know the things you know.
The need for getting stricter is because of the current (and forecasted) increase in new user. This means we can’t afford to become 50% posts that ignore everything our community has already figured out.
LessWrong is an internet forum, but it’s in the direction of a university/academic publication, and such publications only work because editors don’t accept everything.
Source?
my guess is that that claim is slightly exaggerated, but I expect sources exist for an only mildly weaker claim. I certainly have been specifically mocked for my username in places that watch this site, for example.
This is an example of people mocking LW for something. Portia is making a claim about LW users’ internal emotional states; she is asserting that they care deeply about academic recognition and feel infuriated they’re not getting it. Does this describe you or the rest of the website, in your experience?
lens portia’s writing out of frustrated tone first and it makes more sense. they’re saying that recognition is something folks care about (yeah, I think so) and aren’t getting to an appropriate degree (also seems true). like I said in my other comment—tone makes it harder to extract intended meaning.
Well, I disagree. I have literally zero interest in currying the favor of academics, and think Portia is projecting a respect and yearning for status within universities onto the rest of us that mostly doesn’t exist. I would additionally prefer if this community were able to set standards for its members without having to worry about or debate whether or not asking people to read canon is a status grab.
sure. I do think it’s helpful to be academically valid sometimes though. you don’t need to care, but some do some of the time somewhat. maybe not as much as the literal wording used here. catch ya later, anyhow.
strong agree, single upvote: harsh tone, but reasonable message. I hope the tone doesn’t lead this point to be ignored, as I do think it’s important. but leading with mocking does seem like it’s probably why others have downvoted. downvote need not indicate refusal to consider, merely negative feedback to tone, but I worry about that, given the agree votes are also in the negative.
Thank you. And I apologise for the tone. I think the back of my mind was haunted by Shoggoth with a Smiley face giving me advice for my weekend plans, and that emotional turmoil came out the wrong way.
I am in the strange position of being on this forum, and in academia, and seeing both sides engage in the same barrier keeping behaviour, and call it out as elitist and misguided in the other but a necessary way to ensure quality and affirm your superior identity in your own group is jarring. I’ve found valuable and admirable practices and insights in both, else I would not be there.
Any group that bears a credential, or performs negative selection of some kind, will bear the traits you speak of. ’Tis the nature of most task-performing groups human society produces. Alas, one cannot escape it, even coming to a group that once claimed to eschew credentialism. Nonetheless, it is still worthwhile to engage with these groups intellectually.
I cannot access www.lesswrong.com/rejectedcontent (404 error). I suspect you guys forgot to give access to non-moderators, or you meant www.lesswrong.com/moderation (But there are no rejected posts there, only comments)
We didn’t build that yet but plan to soon. (I think what happened was Ruby wrote this up in a private google doc, I encouraged him to post it as a comment so I could link to it, and both of us forgot it included that explicit link. Sorry about that, I’ll edit it to clarify)