Archipelago hasn’t worked, but, well “Real Archipelago hasn’t even been attempted”. But I’m not sure it actually helps.
Er… has any ‘Archipelago’ been tried? When you say “Archipelago hasn’t worked”, you’re talking about… what?
Anyhow, as far as your three options go… some pros & cons:
Try to actually resolve the longstanding major disagreements …
Pro: If you succeed, then we march forward into the future in productive harmony! And you (probably) save yourself (and everyone else) a ton of heartache, going forward.
Con: If you fail, then you’ve wasted a ton of effort and accomplished at most nothing, and possibly even made everyone angrier at each other, etc.
Attempt to build real archipelago features …
Pro: Pretty hard to imagine a scenario where you totally waste your time, if you do this (unless you’re, like, such a bad programmer/designer/whatever that you try to build some features but you just fail somehow). In the worst case, you have new features that are useful for something or someone, even if they don’t solve the problem(s) they were meant to solve. And in the best case, you solve all the problems!
Con: Actually maybe the worst case is instead much worse: the new features have an effect but it’s in the opposite direction from what you intended, or there are some horrible consequences you didn’t foresee, etc.
Attempt to mostly bypass that discussion by just focusing on the Open Questions feature-set …
Pro: Similar to above, but best case is not as great (though still good) and worst case is almost certainly not nearly as bad—a lower-variance approach, but still it seems like at worst you’ve got some new features that are useful.
Con: Probably doesn’t do much to solve any of the serious problems. If, once you’ve done this, all the same problems remain, and meanwhile the community has been hemorrhaging participants… haven’t you wasted time that might’ve been better spent solving the aforesaid serious problems?
Something I haven’t actually been clear on re: your opinions:
If LW ended up leaning hard into Archipelago, and if we did something like “posts can be either set to ‘debate’ mode, or ‘collaborative’ mode, or there are epistemic statuses indicating things like “this post is about early stage brainstorming vs this post is ready to be seriously critiqued”,
Does that actually sound good to you?
My model of you was worried that that sort of thing could well result in horrible consequences (via giving bad ideas the ability to gain traction).
(I suppose you might believe that, but still think it’s superior to the status of quo of ‘sorta kinda that but much more confusingly’)
Having good and correct norms on Less Wrong > having some sort of Archipelago, and thereby having good and correct norms on some parts of Less Wrong > having bad and wrong norms everywhere on Less Wrong
We did discuss this a while ago, actually, though I’m afraid I haven’t the time right now to look for the comment thread in question. Simply: if you can set posts to “collaborative mode”, and there’s nothing wrong with that (norm-wise), well, everyone sets their posts to “collaborative mode” all the time (because defending their ideas is hard and annoying), the end. (Unless you also have strong norms along the lines of “using or even mentioning ideas which have thus far been discussed only in ‘collaborative mode’ posts, in other discussions, as if they have been properly defended and are anything but baseless speculation, is a faux pas; conversely, calling out such usage is right and proper and praiseworthy and deserving of upvotes”. But such a norm, which would be very useful and beneficial, nonetheless seems to me to be unlikely to end up as part of the Archipelago you envision. Or am I mistaken, do you think?)
This seems to assume there is one correct set of norms for all conversations. That would be really surprising to me. Do you think there’s one set that is Always Correct, or that the switching costs outweigh the gains from tailored norms?
How much work we take this qualifier to be doing is, of course, a likely point of disagreement, but if you see it as doing most of the work in my comment, then assume that you’ve misunderstood me.
I think a core disagreement here has less to do with collaborative vs debate. Ideas can, and should, be subjected to extreme criticism within a collaborative frame.
My disagreement with your claim is more about how intellectual progress works. I strongly believe you need a several stages, with distinct norms. [Note: I’m not sure these stages listed are exactly right, but think they point roughly in the right direction]
1. Early brainstorming, shower thoughts, and play.
2. Refining brainstormed ideas into something coherent enough to be evaluated
3. Evaluating, and iterating on, those ideas. [It’s around this stage that I think comments like the ones I archetypically associate with you become useful]
4. If an idea seems promising enough to do rigorously check (i.e. something like ’do real science, spending thousands or millions of dollars to run experiments), figure out how to do that. Which is complicated enough that it’s its own step, separate from....
5. Do real science (note: this section is a bit different for things like math and philosophy)
6. If the experiments disconfirm the idea (or, if an earlier stage truncated the idea before you got to the “real science” part), make sure to say “oops”, and make it common knowledge that the idea is wrong.
I think the first two stages are extremely important (and bad things happen when you punish doing it publicly). The last stage is also extremely important. Right now, even at its most rigorous, the pipeline of ideas at LessWrong seems to stop around the 3rd stage.
I don’t expect you to agree with all of that right now, but I am curious: how much would your concerns be addressed if we had clearer/better systems for the final step?
4 and 5 seem hard. Consider the “Archipelago” idea. Also, this model assumes the idea is easily disproved/proved, and isn’t worth iterating on further.
(Rough) Contrasting model:
1) I want to make a [lightbulb] (before lightbulbs have been invented).
2) Come up with a design.
3) Test the design.
4) If it fails, go back to step 2, and start over, or refine the design, and go to step 3.
Repeat 100 times, or until you succeed.
5) If it works, come up with a snazzy name, and start a business.
Er… has any ‘Archipelago’ been tried? When you say “Archipelago hasn’t worked”, you’re talking about… what?
We *did* spend several months working on the Ban user and users-setting-moderation-norms features, and write up a lengthy post discussing how we hoped they would be used, and a couple people very briefly tried using them. So… “any” Archipelago has been tried.
But certainly it was not be tried in a way where the features were clear enough that I’d have expected people to have “really” tried it.
The rest of the pros-and-cons seem relevant, although I’m currently actually more optimistic about Open Questions than Archipelago (partly for unrelated reasons that have to do with why I think Open Questions was high value in the first place.)
I wonder if Archipelago is one of those features that is best tested in the context of a larger userbase. Right now there is barely one “island” worth of users on LW. Maybe users just aren’t numerous enough for people to expect bad experiences in the comments of their posts which would cause them to use advanced moderation features. It’s not necessarily a bad thing that you guys have built advanced moderation features before they were actually needed. But I suspect the current userbase is not big enough to stress test them.
We’ve seen 42 post in the last 7 days, and on average the community makes ~500 comments per week. Just want to clarify on the current size of the LW userbase.
Actually yes. For reasons of time, I won’t write stuff now, but look out for a post in Meta probably Monday/Tuesday, with some thoughts on moving in that direction (and agreeing more with your take here than I did at the time).
I only mention the data because I substantially under-predicted it before Ruby told me what the true numbers were.
Edit: Sorry! Turns out that I won’t be writing this post.
Hmm, indeed. I suppose that does qualify as a form of Archipelago, if looked at in the right way. Those features, and that perspective, didn’t occur to me when I wrote the grandparent, but yes, fair point.
I think we agree w.r.t. “tried, sort of, but not ‘really’”.
To be clear, though – all the features that are necessary for you to set your own preferred norms on your own posts already exist. You can start writing posts and hosting discussions set in whatever frame you want.
The actions available are:
– set your default moderation guidelines in your user profile
– set post-specific moderation guidelines in a given post
– if a user has commented in a way that violates your guidelines, and doesn’t stop after you remind them of them, you can click on a comment’s menu item to delete said comment or ban said user.
So if you do prefer a given style of discourse, you can set that for your own posts, and if you wanted to discuss someone else’s post in a different style of discourse than they prefer, I think it’d be good to create your own thread for doing so.
Note: These features do not seem to exist on GW. (Not that I miss them since I don’t feel a need to use them myself.)
Questions: Is anyone using these features at all? Oh I see you said earlier “a couple people very briefly tried using them”. Do you know why they stopped? Do you think you overestimated how many people would use it, in a way that could have been corrected (for example by surveying potential users or paying more attention to skeptical voices)? (To be fair, upon reviewing the comments on your Archipelago posts, there weren’t that many skeptical voices, although I did upvote this one.) Given that you spend several months on Archipelago, it seems useful to do a quick postmortem on lessons learned?
Each of the features has been used a bit, even recently. (I think there’s 3-7 people who’ve set some kind of intentional moderation style and/or guideline, and at least one person who’s banned a user from their posts recently).
I think the moderation guidelines help to set expectations and the small bit of counterfactual threat of banning helps lend them a bit of force.
The features were also a pre-requisite for Eliezer posting and/or allowing admins to do crossposts on his behalf (I doubt we would have prioritized them as hard without that, although I’d been developing the archipelago-concept-as-applied-to-lesswrong before then)
So I don’t consider the features a failure, so much as “they didn’t have this outsized, qualitatively different benefit” that I was hoping for.
The features were also a pre-requisite for Eliezer posting and/or allowing admins to do crossposts on his behalf (I doubt we would have prioritized them as hard without that, although I’d been developing the archipelago-concept-as-applied-to-lesswrong before then)
Yet Eliezer still isn’t participating on Less Wrong… is there some reason for that? Were the implemented features insufficient? Is there still something left to do?
The moderation tools were a prerequisite even for the degree of Eliezer participation you currently see (where periodically Robby crossposts things on his behalf), which I still consider quite worth it.
As Richard notes, Eliezer isn’t really participating in online discussion these days and that looks unlikely to change.
That makes it even worse, if true! If he doesn’t post anywhere, then he wasn’t ever going to post here, so what in the world was the point of all these changes and features and all that stuff that was allegedly “so that Eliezer would post here”?!
Re: GW – obviously the GW team has limited time, but there shouldn’t be anything stopping them from implementing these features. And in the meanwhile, if you hop over to lesswrong.com to use a feature (such as deleting a comment or banning .a user) it should have the desired effect over on greaterwrong.
I do expect, as the LW team tries more and more experimental things that are designed to radically change the shape of the site, that the GW experience will start to feel a bit confusing, depending on how much time the GW team has to implement things.
[note to GW team: I know at least part of the problem is that the LW team hasn’t been that proactive about communicating our plans. My current impression is that you’re sufficiently bottlenecked on dev-time that doing so wouldn’t really help, but if you thought otherwise I could maybe arrange for that]
One recent example are Related Questions, which I expect to be a major component of how the questions feature (and the site overall) ends up working. The greaterwrong version of this question doesn’t show it’s parent question, either at the top of the page or in a list further down, which changes the context of the question quite a bit. See the lesswrong version).
(Related questions overall are still in a “soft beta” where we’re still tweaking them a bunch and aren’t confident that they’re usable enough to really advertise, but I expect that to change within a couple weeks)
It is true that we’re bottlenecked on developer time, yes. We wouldn’t say no to more communication of the LW team’s plans, of course, but that is indeed not a major problem at this time, as far as I can tell.
One thing that would be quite useful would be a maintained centralized list of LW features (preferably in order of when they were added, and with links to documentation… a Blizzard-style list of “patch notes”, in other words, aggregated into a change history, and kept somewhere central and easy to find).
If, perhaps, this were a post that were to be updated as new features rolled in, we could use it as a way to track GW vs. LW feature parity (via comments and updating of the post itself), and as a publicly visible roadmap for same.
I think the recently published FAQ has almost all of our features, though not in an easily skimmable or accessable format. But definitely better than what we had before it.
Knowing your plans could definitely make a difference—I do want to prioritize fixing any problems that make GW confusing to use, as well as adding features that someone has directly asked for. As such, I just implemented the related questions feature.
I think another major issue is going to be custom commenting-guidelines, which GreaterWrong doesn’t have AFAICT.
Right now, custom commenting guidelines aren’t actually all that clear on LW, and I don’t think people rely on them much. But we’ve been talking about making guidelines and moderation-policies appear next to commenting boxes as soon as you start typing, or otherwise making it more visually distinct what the norms of a given discussion section is.
If we ended up learning harder into the archipelago model, this would become particularly important.
Er… has any ‘Archipelago’ been tried? When you say “Archipelago hasn’t worked”, you’re talking about… what?
Anyhow, as far as your three options go… some pros & cons:
Pro: If you succeed, then we march forward into the future in productive harmony! And you (probably) save yourself (and everyone else) a ton of heartache, going forward.
Con: If you fail, then you’ve wasted a ton of effort and accomplished at most nothing, and possibly even made everyone angrier at each other, etc.
Pro: Pretty hard to imagine a scenario where you totally waste your time, if you do this (unless you’re, like, such a bad programmer/designer/whatever that you try to build some features but you just fail somehow). In the worst case, you have new features that are useful for something or someone, even if they don’t solve the problem(s) they were meant to solve. And in the best case, you solve all the problems!
Con: Actually maybe the worst case is instead much worse: the new features have an effect but it’s in the opposite direction from what you intended, or there are some horrible consequences you didn’t foresee, etc.
Pro: Similar to above, but best case is not as great (though still good) and worst case is almost certainly not nearly as bad—a lower-variance approach, but still it seems like at worst you’ve got some new features that are useful.
Con: Probably doesn’t do much to solve any of the serious problems. If, once you’ve done this, all the same problems remain, and meanwhile the community has been hemorrhaging participants… haven’t you wasted time that might’ve been better spent solving the aforesaid serious problems?
Something I haven’t actually been clear on re: your opinions:
If LW ended up leaning hard into Archipelago, and if we did something like “posts can be either set to ‘debate’ mode, or ‘collaborative’ mode, or there are epistemic statuses indicating things like “this post is about early stage brainstorming vs this post is ready to be seriously critiqued”,
Does that actually sound good to you?
My model of you was worried that that sort of thing could well result in horrible consequences (via giving bad ideas the ability to gain traction).
(I suppose you might believe that, but still think it’s superior to the status of quo of ‘sorta kinda that but much more confusingly’)
Having good and correct norms on Less Wrong > having some sort of Archipelago, and thereby having good and correct norms on some parts of Less Wrong > having bad and wrong norms everywhere on Less Wrong
We did discuss this a while ago, actually, though I’m afraid I haven’t the time right now to look for the comment thread in question. Simply: if you can set posts to “collaborative mode”, and there’s nothing wrong with that (norm-wise), well, everyone sets their posts to “collaborative mode” all the time (because defending their ideas is hard and annoying), the end. (Unless you also have strong norms along the lines of “using or even mentioning ideas which have thus far been discussed only in ‘collaborative mode’ posts, in other discussions, as if they have been properly defended and are anything but baseless speculation, is a faux pas; conversely, calling out such usage is right and proper and praiseworthy and deserving of upvotes”. But such a norm, which would be very useful and beneficial, nonetheless seems to me to be unlikely to end up as part of the Archipelago you envision. Or am I mistaken, do you think?)
Nod. I do think the failure mode your pointing at is an important thing for the system to address.
This seems to assume there is one correct set of norms for all conversations. That would be really surprising to me. Do you think there’s one set that is Always Correct, or that the switching costs outweigh the gains from tailored norms?
All conversations? Certainly not. All conversations on Less Wrong? To a first approximation[1], yes.
How much work we take this qualifier to be doing is, of course, a likely point of disagreement, but if you see it as doing most of the work in my comment, then assume that you’ve misunderstood me.
I think a core disagreement here has less to do with collaborative vs debate. Ideas can, and should, be subjected to extreme criticism within a collaborative frame.
My disagreement with your claim is more about how intellectual progress works. I strongly believe you need a several stages, with distinct norms. [Note: I’m not sure these stages listed are exactly right, but think they point roughly in the right direction]
1. Early brainstorming, shower thoughts, and play.
2. Refining brainstormed ideas into something coherent enough to be evaluated
3. Evaluating, and iterating on, those ideas. [It’s around this stage that I think comments like the ones I archetypically associate with you become useful]
4. If an idea seems promising enough to do rigorously check (i.e. something like ’do real science, spending thousands or millions of dollars to run experiments), figure out how to do that. Which is complicated enough that it’s its own step, separate from....
5. Do real science (note: this section is a bit different for things like math and philosophy)
6. If the experiments disconfirm the idea (or, if an earlier stage truncated the idea before you got to the “real science” part), make sure to say “oops”, and make it common knowledge that the idea is wrong.
I think the first two stages are extremely important (and bad things happen when you punish doing it publicly). The last stage is also extremely important. Right now, even at its most rigorous, the pipeline of ideas at LessWrong seems to stop around the 3rd stage.
I don’t expect you to agree with all of that right now, but I am curious: how much would your concerns be addressed if we had clearer/better systems for the final step?
4 and 5 seem hard. Consider the “Archipelago” idea. Also, this model assumes the idea is easily disproved/proved, and isn’t worth iterating on further.
(Rough) Contrasting model:
1) I want to make a [lightbulb] (before lightbulbs have been invented).
2) Come up with a design.
3) Test the design.
4) If it fails, go back to step 2, and start over, or refine the design, and go to step 3.
Repeat 100 times, or until you succeed.
5) If it works, come up with a snazzy name, and start a business.
We *did* spend several months working on the Ban user and users-setting-moderation-norms features, and write up a lengthy post discussing how we hoped they would be used, and a couple people very briefly tried using them. So… “any” Archipelago has been tried.
But certainly it was not be tried in a way where the features were clear enough that I’d have expected people to have “really” tried it.
The rest of the pros-and-cons seem relevant, although I’m currently actually more optimistic about Open Questions than Archipelago (partly for unrelated reasons that have to do with why I think Open Questions was high value in the first place.)
I wonder if Archipelago is one of those features that is best tested in the context of a larger userbase. Right now there is barely one “island” worth of users on LW. Maybe users just aren’t numerous enough for people to expect bad experiences in the comments of their posts which would cause them to use advanced moderation features. It’s not necessarily a bad thing that you guys have built advanced moderation features before they were actually needed. But I suspect the current userbase is not big enough to stress test them.
We’ve seen 42 post in the last 7 days, and on average the community makes ~500 comments per week. Just want to clarify on the current size of the LW userbase.
Thanks for the data! Any thoughts on this Wei Dai comment?
Actually yes. For reasons of time, I won’t write stuff now, but look out for a post in Meta probably Monday/Tuesday, with some thoughts on moving in that direction (and agreeing more with your take here than I did at the time).
I only mention the data because I substantially under-predicted it before Ruby told me what the true numbers were.
Edit: Sorry! Turns out that I won’t be writing this post.
What happened?
The team decided to hold off on publishing some thoughts for awhile, sorry about that.
Hmm, indeed. I suppose that does qualify as a form of Archipelago, if looked at in the right way. Those features, and that perspective, didn’t occur to me when I wrote the grandparent, but yes, fair point.
I think we agree w.r.t. “tried, sort of, but not ‘really’”.
To be clear, though – all the features that are necessary for you to set your own preferred norms on your own posts already exist. You can start writing posts and hosting discussions set in whatever frame you want.
The actions available are:
– set your default moderation guidelines in your user profile
– set post-specific moderation guidelines in a given post
– if a user has commented in a way that violates your guidelines, and doesn’t stop after you remind them of them, you can click on a comment’s menu item to delete said comment or ban said user.
So if you do prefer a given style of discourse, you can set that for your own posts, and if you wanted to discuss someone else’s post in a different style of discourse than they prefer, I think it’d be good to create your own thread for doing so.
Note: These features do not seem to exist on GW. (Not that I miss them since I don’t feel a need to use them myself.)
Questions: Is anyone using these features at all? Oh I see you said earlier “a couple people very briefly tried using them”. Do you know why they stopped? Do you think you overestimated how many people would use it, in a way that could have been corrected (for example by surveying potential users or paying more attention to skeptical voices)? (To be fair, upon reviewing the comments on your Archipelago posts, there weren’t that many skeptical voices, although I did upvote this one.) Given that you spend several months on Archipelago, it seems useful to do a quick postmortem on lessons learned?
Each of the features has been used a bit, even recently. (I think there’s 3-7 people who’ve set some kind of intentional moderation style and/or guideline, and at least one person who’s banned a user from their posts recently).
I think the moderation guidelines help to set expectations and the small bit of counterfactual threat of banning helps lend them a bit of force.
The features were also a pre-requisite for Eliezer posting and/or allowing admins to do crossposts on his behalf (I doubt we would have prioritized them as hard without that, although I’d been developing the archipelago-concept-as-applied-to-lesswrong before then)
So I don’t consider the features a failure, so much as “they didn’t have this outsized, qualitatively different benefit” that I was hoping for.
Yet Eliezer still isn’t participating on Less Wrong… is there some reason for that? Were the implemented features insufficient? Is there still something left to do?
The moderation tools were a prerequisite even for the degree of Eliezer participation you currently see (where periodically Robby crossposts things on his behalf), which I still consider quite worth it.
As Richard notes, Eliezer isn’t really participating in online discussion these days and that looks unlikely to change.
Does Eliezer post anywhere public these days? His postings to Facebook are infrequent, and I don’t know of him posting anywhere else.
That makes it even worse, if true! If he doesn’t post anywhere, then he wasn’t ever going to post here, so what in the world was the point of all these changes and features and all that stuff that was allegedly “so that Eliezer would post here”?!
He seems to post on Twitter pretty frequently...
¯\_(ツ)_/¯
Re: GW – obviously the GW team has limited time, but there shouldn’t be anything stopping them from implementing these features. And in the meanwhile, if you hop over to lesswrong.com to use a feature (such as deleting a comment or banning .a user) it should have the desired effect over on greaterwrong.
I do expect, as the LW team tries more and more experimental things that are designed to radically change the shape of the site, that the GW experience will start to feel a bit confusing, depending on how much time the GW team has to implement things.
[note to GW team: I know at least part of the problem is that the LW team hasn’t been that proactive about communicating our plans. My current impression is that you’re sufficiently bottlenecked on dev-time that doing so wouldn’t really help, but if you thought otherwise I could maybe arrange for that]
One recent example are Related Questions, which I expect to be a major component of how the questions feature (and the site overall) ends up working. The greaterwrong version of this question doesn’t show it’s parent question, either at the top of the page or in a list further down, which changes the context of the question quite a bit. See the lesswrong version).
(Related questions overall are still in a “soft beta” where we’re still tweaking them a bunch and aren’t confident that they’re usable enough to really advertise, but I expect that to change within a couple weeks)
It is true that we’re bottlenecked on developer time, yes. We wouldn’t say no to more communication of the LW team’s plans, of course, but that is indeed not a major problem at this time, as far as I can tell.
One thing that would be quite useful would be a maintained centralized list of LW features (preferably in order of when they were added, and with links to documentation… a Blizzard-style list of “patch notes”, in other words, aggregated into a change history, and kept somewhere central and easy to find).
If, perhaps, this were a post that were to be updated as new features rolled in, we could use it as a way to track GW vs. LW feature parity (via comments and updating of the post itself), and as a publicly visible roadmap for same.
I think the recently published FAQ has almost all of our features, though not in an easily skimmable or accessable format. But definitely better than what we had before it.
Agree having a proper list would be good.
Knowing your plans could definitely make a difference—I do want to prioritize fixing any problems that make GW confusing to use, as well as adding features that someone has directly asked for. As such, I just implemented the related questions feature.
Thanks! (missed this the first time around)
I think another major issue is going to be custom commenting-guidelines, which GreaterWrong doesn’t have AFAICT.
Right now, custom commenting guidelines aren’t actually all that clear on LW, and I don’t think people rely on them much. But we’ve been talking about making guidelines and moderation-policies appear next to commenting boxes as soon as you start typing, or otherwise making it more visually distinct what the norms of a given discussion section is.
If we ended up learning harder into the archipelago model, this would become particularly important.
Yup. This post is essentially the result of that post-mortem.