(I prefer to ignore the specific details of the implementation and only discuss the idea in general. Because debating the details feels like “too soon; I need to be sure about the higher level first”.)
There is a profound difference between building “commons of knowledge” and a “discussion website”. In some sense, they are the opposite of each other. Discussion websites, by their nature, attract people who have a preference for discussion. Curiosity seeks to annihilate itself, but social interaction seeks to grow and persist. And if people who feel that the debate is no longer productive start leaving, well, those who don’t mind will become the new norm.
But how do you build the commons of knowledge without having the discussion first? How would people contribute? How would they point out the mistakes? How would they conclude whether the supposed “mistakes” are actual mistakes or not? How would they coordinate in time, so that the published piece of knowledge is reviewed now, and the author can read the responses and update the text now? (as opposed to e.g. someone randomly pointing out an error ten years later, and the original author no longer being active)
Seems to me that successful knowledge-oriented websites, such as Wikipedia and Stack Exchange, solve this problem by separating the knowledge from the discussion. There are different ways to do it.
On Wikipedia, the article presents the “final answer”, and the whole discussion happens on the Talk page. (There is typically one Talk page per article, but if the discussion grows unusually long, the older parts are gradually moved into separate read-only Archive pages.) If someone believes they have found a mistake in the article, they edit the article, and optionally explain the reason on the Talk page. If someone else believes they were wrong, they revert the changes in the article, and optionally explain their reasons on the Talk page. This sometimes leads to an “edit war”, when various solutions are implemented, for example the page can be “locked” so the ordinary users can’t edit it anymore, and all they can do is argue for their case on the Talk page; only privileged remain able to edit the article.
On Stack Exchange, each user can provide their own answer to the question, and all answers are displayed below the question; the ones with the most votes are displayed first. There is further an option to write a short comment below the question or an answer, and to upvote the comment. (The comment structure is linear, not hierarchical.) To prevent the growth of the discussion, only a limited amount of text is displayed immediately; displaying the remaining text requires further user action. For example, if there are more than N answers, only the highest upvoted N answers are displayed below the question; the remaining ones are moved to page 2 or more. Similarly, if there are more than M comments below a question or an answer, only the highest upvoted M comments are displayed, unless the user explicitly clicks “show more comments”. Together this means that regardless of the number of answers and the number of comments, the user clicking on the question still receives a relatively short page, containing the most relevant things. Furthermore, debate that is not strictly on-topic is discouraged, the content of such debate is frequently removed, and the users are advised to debate their opinions on a chat, outside the question-and-answers area.
We can’t copy either model directly. Less Wrong is neither an encyclopedia nor a Q&A website. (We discuss things that sometimes don’t have official names yet; and we often provide insights and opinions without being asked first.) We are more like a blog, or maybe a news site. But blogs are personal (worked well while Eliezer was writing the original Sequences) or for a predefined group of authors; they don’t scale well. A the purpose of news is typically to maximize paperclips… ahem, pageviews… which is best achieved by techniques contrary to the goal of developing and spreading rationality. (I suspect that the next step in the online news business will be websites with fake news generated automatically using machine learning and A/B testing. Using thousands of different domain names, to make it impossible to write a domain-name filter; generating enough content to keep all domains active and seemingly different. Also, automatically generated comments below the articles. And automatically generated replies to humans who get fooled by the system.)
Seems to me these would be reasonable guidelines for a good solution:
Keep the “final product” and the “intermediate products” unambiguously separated, in a way that clearly communicates to the reader’s System 1. Preferably, each of them on a different domain name, with different design. Separate “LW, the sacred tome of rationalist knowledge” from “LW, where highly intelligent people procrastinate”.
The change from the “intermediate” to the “final” state must always be done by a conscious decision of a group of trusted people. (It is not necessary to make all of them approve each change, more like: any two or three of them may promote the content, and then any one of them can veto the change, and if that happens, then all of them will debate behind the closed doors until they reach a consensus.) No amount of votes is enough to automatically promote an article; and no amount of karma is enough to automatically promote a person to the trusted circle. On the other hand, members of the trusted circle are publicly known. And within the trusted circle, they can review all other other’s actions and votes.
Changing the article from the “intermediate” to the “final” state may include asking the original author to rewrite it (or to consent with someone else’s rewrite). Thus we would avoid the dilemma of “some part of this article are low-quality and lenghty, but here are a few really valuable points, so maybe the cost-benefit analysis suggests we promote it anyway”. Nope, just rewrite it to keep the good parts and remove the bad parts; it will take you an afternoon, and the updated version will be there displayed to everyone for years.
Promoting the article to the “final” state does not mean promoting comments below it to the “final” states. First, no comments should get promoted automatically; and second, most comments should not be promoted at all. So, the “final” website should just display the article without the comments (but maybe with a link to the original discussion). If some comments are considered worthy enough to be canonized along with the article, they could become a part of the article itself, e.g. either by fixing the mistake, or adding a footnote. Paraphrasing Dijkstra, we should not regard LW comments as “lines produced” but as “lines spent”.
New visitors to the website should be directed to the “final” version foremost, and to the “intermediate” version only as a secondary option. So unless they have a specific plan, they will read some parts of the “final” version before they start participating in the “intermediate” section. In other words, having people debating on LW without reading the Sequences first is a result of bad web design.
EDIT:
In some sense, having LW separated into forum and wiki is a partial step in this direction, but it differs from this proposal in many important ways:
The highest-value content stays in the forum section, along with the low-value content. (It is merely linked from the wiki.) Thus, at least to my System 1, the good articles and the bad articles are connected more than any of them is with the wiki. Also, all comments, whether good or bad, remain connected with the good articles.
When new people visit the page for the first time, the first thing they see is… a description that seems like a Wikipedia article stub; four recent articles; and four non-recent articles. Not good. The title page should contain at least 95% of nicely designed hand-picked best content, and only a small link towards the discussion somewhere in the corner.
On the other hand, a technically simple implementation of my suggestion would be to create another website—which could be technically a wiki, but it absolutely shouldn’t look like one; all the wiki-related buttons should be hidden from the average observer, and only available for the “inner circle”—and copy the hand-picked best articles there. For the average visitor, there would be no way to post anything; only to click the links and read the contents. Occassionally, a link would point them towards a relevant discussion on LW, but to their System 1 that would be obviously a link to an external website, with a different design. (The “inner circle” would have a Slack debate somewhere, where they would talk about which articles deserve to be copied. All contributors to LW would have to consent in advance with the possibiity of having their article copied to the other website—and removed from LW, to avoid having duplicates.)
I think your ideas are very compatible with my existing proposal!
I agree about the “too soon” aspect, but this basically came to me fully formed, and it wasn’t clear to me that teasing out a part of it to present instead of presenting it all was the right thing. Maybe I should have held off on proposing solutions.
Well, I am guilty of proposing a solution soon, too. But it’s interesting to see (ignoring the minor details), where we agree, and where our proposals differ. This is a quick comparision:
Common points:
software change is necessary;
creating a personalized “bubble” is not the direction we want to go, because we want to create commons;
but a democracy where all randos from internet have an equal vote is also not the way (even ignoring the issue of sockpuppets);
so we needs a group of people with trusted opinion;
a binary decision of what content is “really in” could help people who only want to spend short time online (and we should optimize for these, or at least not actively against them);
a lot of good new content is likely to appear in the “experimental” part (that’s where all the new talents are), from where it should be promited to the “in” part;
this promotion of content should be done only by the people who are already “in” (but the opinion of others can be used as a hint).
Differences:
scalable N-tiered system (reflecting familiarity and compatibility with the existing commons); or
just a “user tier” and “admin tier” with quite different rules;
content filtered by a slider (“in” or “in + somewhat less in” etc.); or
the “in” content and the “experimental / debate” content separated from each other as much as possible;
a voting system where votes from higher tiers can override the votes from lower tiers; or
probably something pretty similar, except that the “promote or not” decision would be “two or three admins say yes, and either no admin says no, or the dictator overrides”.
The summary of our differences seems to be that you want to generalize to N tiers, which all use the same algorithm; while I assume two groups using dramatically different rules. (Most of the other differences are just further consequences of this one.)
The reason for my side is that I assume that the “trusted group” will be relatively small and busy, for various reasons. (Being instrumentally rational correlates with “having a lot of high-priority work other than voting in LW debates”. Some of the trusted users will also be coders who will be busy fixing and contributing to LW code. And they will have to solve other problems that appear.) I imagine something like about 20 people, of whom only about 5 will actually be active at any given week, and 3 of those will be solving some technical or other problem. In other words, a group too small to need have their mutual communication solved by an algorithm. (And in case of admin conflict, we have the dictator anyway.)
Hi Villiam, your idea sounds like an academic community around rationality. You can think of the discussions as like the events at a conference or workshop where half-baked ideas are thrown about. And you can think of the “final” tome of knowledge as the proceedings of the journal: when an idea has been workshopped enough, it is revised and then published in the journal as Currently Definitive Knowledge.
This framing suggests having a rotating board of editors and a formal peer review system as is common in academic journals.
This is a really great summary. Maybe we should Skype or something to drill down further on our disagreement? Maybe when I’m in London, and so closer to you in timezone?
(I prefer to ignore the specific details of the implementation and only discuss the idea in general. Because debating the details feels like “too soon; I need to be sure about the higher level first”.)
There is a profound difference between building “commons of knowledge” and a “discussion website”. In some sense, they are the opposite of each other. Discussion websites, by their nature, attract people who have a preference for discussion. Curiosity seeks to annihilate itself, but social interaction seeks to grow and persist. And if people who feel that the debate is no longer productive start leaving, well, those who don’t mind will become the new norm.
But how do you build the commons of knowledge without having the discussion first? How would people contribute? How would they point out the mistakes? How would they conclude whether the supposed “mistakes” are actual mistakes or not? How would they coordinate in time, so that the published piece of knowledge is reviewed now, and the author can read the responses and update the text now? (as opposed to e.g. someone randomly pointing out an error ten years later, and the original author no longer being active)
Seems to me that successful knowledge-oriented websites, such as Wikipedia and Stack Exchange, solve this problem by separating the knowledge from the discussion. There are different ways to do it.
On Wikipedia, the article presents the “final answer”, and the whole discussion happens on the Talk page. (There is typically one Talk page per article, but if the discussion grows unusually long, the older parts are gradually moved into separate read-only Archive pages.) If someone believes they have found a mistake in the article, they edit the article, and optionally explain the reason on the Talk page. If someone else believes they were wrong, they revert the changes in the article, and optionally explain their reasons on the Talk page. This sometimes leads to an “edit war”, when various solutions are implemented, for example the page can be “locked” so the ordinary users can’t edit it anymore, and all they can do is argue for their case on the Talk page; only privileged remain able to edit the article.
On Stack Exchange, each user can provide their own answer to the question, and all answers are displayed below the question; the ones with the most votes are displayed first. There is further an option to write a short comment below the question or an answer, and to upvote the comment. (The comment structure is linear, not hierarchical.) To prevent the growth of the discussion, only a limited amount of text is displayed immediately; displaying the remaining text requires further user action. For example, if there are more than N answers, only the highest upvoted N answers are displayed below the question; the remaining ones are moved to page 2 or more. Similarly, if there are more than M comments below a question or an answer, only the highest upvoted M comments are displayed, unless the user explicitly clicks “show more comments”. Together this means that regardless of the number of answers and the number of comments, the user clicking on the question still receives a relatively short page, containing the most relevant things. Furthermore, debate that is not strictly on-topic is discouraged, the content of such debate is frequently removed, and the users are advised to debate their opinions on a chat, outside the question-and-answers area.
We can’t copy either model directly. Less Wrong is neither an encyclopedia nor a Q&A website. (We discuss things that sometimes don’t have official names yet; and we often provide insights and opinions without being asked first.) We are more like a blog, or maybe a news site. But blogs are personal (worked well while Eliezer was writing the original Sequences) or for a predefined group of authors; they don’t scale well. A the purpose of news is typically to maximize paperclips… ahem, pageviews… which is best achieved by techniques contrary to the goal of developing and spreading rationality. (I suspect that the next step in the online news business will be websites with fake news generated automatically using machine learning and A/B testing. Using thousands of different domain names, to make it impossible to write a domain-name filter; generating enough content to keep all domains active and seemingly different. Also, automatically generated comments below the articles. And automatically generated replies to humans who get fooled by the system.)
Seems to me these would be reasonable guidelines for a good solution:
Keep the “final product” and the “intermediate products” unambiguously separated, in a way that clearly communicates to the reader’s System 1. Preferably, each of them on a different domain name, with different design. Separate “LW, the sacred tome of rationalist knowledge” from “LW, where highly intelligent people procrastinate”.
The change from the “intermediate” to the “final” state must always be done by a conscious decision of a group of trusted people. (It is not necessary to make all of them approve each change, more like: any two or three of them may promote the content, and then any one of them can veto the change, and if that happens, then all of them will debate behind the closed doors until they reach a consensus.) No amount of votes is enough to automatically promote an article; and no amount of karma is enough to automatically promote a person to the trusted circle. On the other hand, members of the trusted circle are publicly known. And within the trusted circle, they can review all other other’s actions and votes.
Changing the article from the “intermediate” to the “final” state may include asking the original author to rewrite it (or to consent with someone else’s rewrite). Thus we would avoid the dilemma of “some part of this article are low-quality and lenghty, but here are a few really valuable points, so maybe the cost-benefit analysis suggests we promote it anyway”. Nope, just rewrite it to keep the good parts and remove the bad parts; it will take you an afternoon, and the updated version will be there displayed to everyone for years.
Promoting the article to the “final” state does not mean promoting comments below it to the “final” states. First, no comments should get promoted automatically; and second, most comments should not be promoted at all. So, the “final” website should just display the article without the comments (but maybe with a link to the original discussion). If some comments are considered worthy enough to be canonized along with the article, they could become a part of the article itself, e.g. either by fixing the mistake, or adding a footnote. Paraphrasing Dijkstra, we should not regard LW comments as “lines produced” but as “lines spent”.
New visitors to the website should be directed to the “final” version foremost, and to the “intermediate” version only as a secondary option. So unless they have a specific plan, they will read some parts of the “final” version before they start participating in the “intermediate” section. In other words, having people debating on LW without reading the Sequences first is a result of bad web design.
EDIT:
In some sense, having LW separated into forum and wiki is a partial step in this direction, but it differs from this proposal in many important ways:
The highest-value content stays in the forum section, along with the low-value content. (It is merely linked from the wiki.) Thus, at least to my System 1, the good articles and the bad articles are connected more than any of them is with the wiki. Also, all comments, whether good or bad, remain connected with the good articles.
When new people visit the page for the first time, the first thing they see is… a description that seems like a Wikipedia article stub; four recent articles; and four non-recent articles. Not good. The title page should contain at least 95% of nicely designed hand-picked best content, and only a small link towards the discussion somewhere in the corner.
On the other hand, a technically simple implementation of my suggestion would be to create another website—which could be technically a wiki, but it absolutely shouldn’t look like one; all the wiki-related buttons should be hidden from the average observer, and only available for the “inner circle”—and copy the hand-picked best articles there. For the average visitor, there would be no way to post anything; only to click the links and read the contents. Occassionally, a link would point them towards a relevant discussion on LW, but to their System 1 that would be obviously a link to an external website, with a different design. (The “inner circle” would have a Slack debate somewhere, where they would talk about which articles deserve to be copied. All contributors to LW would have to consent in advance with the possibiity of having their article copied to the other website—and removed from LW, to avoid having duplicates.)
I think your ideas are very compatible with my existing proposal!
I agree about the “too soon” aspect, but this basically came to me fully formed, and it wasn’t clear to me that teasing out a part of it to present instead of presenting it all was the right thing. Maybe I should have held off on proposing solutions.
Well, I am guilty of proposing a solution soon, too. But it’s interesting to see (ignoring the minor details), where we agree, and where our proposals differ. This is a quick comparision:
Common points:
software change is necessary;
creating a personalized “bubble” is not the direction we want to go, because we want to create commons;
but a democracy where all randos from internet have an equal vote is also not the way (even ignoring the issue of sockpuppets);
so we needs a group of people with trusted opinion;
a binary decision of what content is “really in” could help people who only want to spend short time online (and we should optimize for these, or at least not actively against them);
a lot of good new content is likely to appear in the “experimental” part (that’s where all the new talents are), from where it should be promited to the “in” part;
this promotion of content should be done only by the people who are already “in” (but the opinion of others can be used as a hint).
Differences:
scalable N-tiered system (reflecting familiarity and compatibility with the existing commons); or
just a “user tier” and “admin tier” with quite different rules;
content filtered by a slider (“in” or “in + somewhat less in” etc.); or
the “in” content and the “experimental / debate” content separated from each other as much as possible;
a voting system where votes from higher tiers can override the votes from lower tiers; or
probably something pretty similar, except that the “promote or not” decision would be “two or three admins say yes, and either no admin says no, or the dictator overrides”.
The summary of our differences seems to be that you want to generalize to N tiers, which all use the same algorithm; while I assume two groups using dramatically different rules. (Most of the other differences are just further consequences of this one.)
The reason for my side is that I assume that the “trusted group” will be relatively small and busy, for various reasons. (Being instrumentally rational correlates with “having a lot of high-priority work other than voting in LW debates”. Some of the trusted users will also be coders who will be busy fixing and contributing to LW code. And they will have to solve other problems that appear.) I imagine something like about 20 people, of whom only about 5 will actually be active at any given week, and 3 of those will be solving some technical or other problem. In other words, a group too small to need have their mutual communication solved by an algorithm. (And in case of admin conflict, we have the dictator anyway.)
Hi Villiam, your idea sounds like an academic community around rationality. You can think of the discussions as like the events at a conference or workshop where half-baked ideas are thrown about. And you can think of the “final” tome of knowledge as the proceedings of the journal: when an idea has been workshopped enough, it is revised and then published in the journal as Currently Definitive Knowledge.
This framing suggests having a rotating board of editors and a formal peer review system as is common in academic journals.
Seems like a “covergent evolution” of ideas. Many people faced similar problems, and devised similar solutions.
This is a really great summary. Maybe we should Skype or something to drill down further on our disagreement? Maybe when I’m in London, and so closer to you in timezone?
Generally yes; details in PM.
This is a really really good comment. Check out https://arbital.com and let me know what you think. :)