I think you are wrong about the need for a single Schelling point and I submit as evidence: Crony Beliefs. We have a mesh network where valuable articles do get around. Lesswrong is very much visited by many (as evidence by the comments on this post). When individuals judge information worthy; it makes its way around the network and is added to our history.
So: this is subtle. But to my mind, the main issue isn’t that ideas won’t mostly-percolate. (Yes, lots of folks seem to be referring to Crony Beliefs. Yes, Molloch. Yes, etc.) It’s rather that there isn’t a process for: creating common knowledge that an idea has percolated; having people feel empowered to author a reply to an idea (e.g., pointing out an apparent error in its arguments) while having faith that if their argument is clear and correct, others will force the original author to eventually reply; creating a common core of people who have a common core of arguments/analysis/evidence they can take for granted (as with Eliezer’s Sequences), etc.
I’m not sure how to fully explicitly model it. But it’s not mostly about the odds that a given post will spread (let’s call that probability “p”). It’s more about a bunch of second-order effects from thingies requiring that p^4 or something be large (e.g., that you will both have read the post I want to reference (p), and I’ll know you’ll have read it (~p^2), and that that’ll be true for a large enough fraction of my audience that I don’t have to painfully write my post to avoid being misunderstood by the people who haven’t read that one post (maybe ~p^3 or something, depending on threshold proportion), for each of the “that one posts” that I want to reference (again, some slightly higher conjunctive requirement, with the probability correspondingly going down)...
I think I understand what you mean. On one hand it is great to have this fluid network of rationalist websites where everyone chooses the content they prefer to read. We don’t have a single point of failure. We can try different writing styles, different moderation styles, etc. The rationalist community can survive and generate new interesting content even when LW is dying and infested by downvoting sockpuppets, and Eliezer keeps posting kitten videos on Facebook (just kidding).
On the other hand, it is also great to have a shared vocabulary; a list of words I can use freely without having to explain them. Because inferential distance is a thing. (For example, LW allows me to type “inferential distance” without having to explain. Maybe I could just use a hyperlink to the origin of the term. But doing it outside of LW includes a risk of people starting to debate the concept of the “inferential distance” itself, derailing the discussion.) The opposite of public knowledge is the Eternal September.
Maybe “Moloch” is an example that meaningful terms will spread across rationalist websites. (Natural selection of rationalist memes?) Maybe hyperlinking the original source is all it takes; linking to SSC is not more difficult than linking to LW Sequences, or Wikipedia. That is, assuming that the concept is clearly explained in one self-contained article. Which is not always the case.
Consider “motte and bailey”. I consider it a critical rationalist concept, almost as important as “a map is not the territory”. (Technically speaking, it is a narrower version of “a map is not the territory”.) I believe it helps me to see more clearly through most political debates, but it can also be applied outside of politics. And what is the canonical link? Oh, this. So, imagine that I am talking with people who are not regular SSC readers, and we are debating something either unrelated to politics, or at least unrelated to the part of politics that the SSC article talks about, but somehow there appears to be a confusion, which could be easily solved by pointing out that this is yet another instance of the “motte and bailey” fallacy, so I just use these words in a sentence, and provide a hyperlink-explanation to the SSC article. What could possibly go wrong? How could it possibly derail the whole debate?
Okay, maybe the situation with “motte and bailey” could be solved by writing a more neutral article (containing a link to the original article) afterwards, and referring to the neutral article. More generally, maybe we could just maintain a separate Dictionary of Terms Generally Considered Useful by the Rationalist Community. Or maybe the dictionary would suffer the same fate as the Sequences; it would exist, but most new people would completely ignore it, simply because it isn’t standing in the middle of the traffic.
So I guess there needs to be a community which has a community norm of “you must read this information, or else you are not a valid member of this community”. Sounds ugly, when I put it like this, but the opposite is the information just being somewhere without people being able to use it freely in a debate.
My problem with the “shared vocabulary” is that as you note yourself here, it implies that something has already been thought through, and it assumes that you have understood the thing properly. So for example if you reject an argument because “that’s an example of a motte and bailey fallacy”, then this only works if it is in fact correct to reject arguments for that reason.
And I don’t think it is correct. One reason why people use a motte and bailey is that they are looking for some common ground with their interlocutor. Take one of Scott’s examples, with this motte and bailey:
God is just the order and love in the universe
God is an extremely powerful supernatural being who punishes my enemies
When the person asserts #1, it is not because they do not believe #2. It is because they are looking for some partial expression of their belief that the other person might accept. In their understanding, the two statements do not contradict one another, even though obviously the second claims a good deal more than the first.
Now Scott says that #1 is “useless,” namely that even if he could theoretically accept the word “God” as applying to this, there is no reason for him to do this, because there is nowhere to go from there. And this might be true. But the fact that #2 is false does not prove that it is true. Most likely, if you work hard, you can find some #3, stronger than #1, but weaker than #2, which will also be defensible.
And it would be right to tell them to do the work that is needed. But it would be wrong to simply say, “Oh, that’s a motte and bailey” and walk away.
This is not merely a criticism of this bit of shared vocabulary, so that it would just be a question of getting the right shared vocabulary. A similar criticism will apply to virtually any possible piece of shared vocabulary—you are always assuming things just by using the vocabulary, and you might be wrong in those assumptions.
Making shared vocabulary common and explicit usually makes it faster to iterate. For example, the EA community converged on the idea of replaceability as an important heuristic for career decisions for a while, and then realized that they’d been putting too much emphasis there and explicitly toned it down. But the general concept had been floating around in discussion space already, giving it a name just made it easier to explicitly think about.
I think I agree with this in one sense and disagree in another. In particular, in regard to “giving it a name just made it easier to explicitly think about” :
I agree that this makes it easier to reason about, and therefore you might come to conclusions faster and so on, even correctly.
I don’t agree that we really made it easier to think about. What we actually did is make it less necessary to think about it at all, in order to come to conclusions. You can see how this works in mathematics, for example. One of the main purpose of the symbols is to abbreviate complicated concepts so that you don’t have to think through them every time they come up.
I think the second point here is also related to my objection in the previous comment. However, overall, the first point might be overall more important, so that the benefit outweighs the costs, especially in terms of benefit to a community.
What are you using this word to mean? At a guess it sounds like, “ideas will float to the surface” but also it does not always mean that, as used in “has percolated”. Percolate relates to filtering of a substance like coffee, to get the good bits from the bad. Can you repeat the above without using this word?
Are we looking to separate and elevate good ideas from the general noise on the interwebs, or are we looking to ensure ideas filter through the diaspora to every little sub group that exists? Or are we looking to filter something else? I am not sure which you are trying to describe.
If you want to describe an earlier post that is well know, and well spread, it should be enough to describe the name of the concept, i.e. crony beliefs. If you want to reference a less well known concept; it should be enough to name the author and link to their post, like if I wanted to refer to the list of common human goals and talk about things that relate to it.
I don’t see the gravity of the problem you are trying to describe with your concerns.
So: this is subtle. But to my mind, the main issue isn’t that ideas won’t mostly-percolate. (Yes, lots of folks seem to be referring to Crony Beliefs. Yes, Molloch. Yes, etc.) It’s rather that there isn’t a process for: creating common knowledge that an idea has percolated; having people feel empowered to author a reply to an idea (e.g., pointing out an apparent error in its arguments) while having faith that if their argument is clear and correct, others will force the original author to eventually reply; creating a common core of people who have a common core of arguments/analysis/evidence they can take for granted (as with Eliezer’s Sequences), etc.
I’m not sure how to fully explicitly model it. But it’s not mostly about the odds that a given post will spread (let’s call that probability “p”). It’s more about a bunch of second-order effects from thingies requiring that p^4 or something be large (e.g., that you will both have read the post I want to reference (p), and I’ll know you’ll have read it (~p^2), and that that’ll be true for a large enough fraction of my audience that I don’t have to painfully write my post to avoid being misunderstood by the people who haven’t read that one post (maybe ~p^3 or something, depending on threshold proportion), for each of the “that one posts” that I want to reference (again, some slightly higher conjunctive requirement, with the probability correspondingly going down)...
I wish I knew how to model this more coherently.
I think I understand what you mean. On one hand it is great to have this fluid network of rationalist websites where everyone chooses the content they prefer to read. We don’t have a single point of failure. We can try different writing styles, different moderation styles, etc. The rationalist community can survive and generate new interesting content even when LW is dying and infested by downvoting sockpuppets, and Eliezer keeps posting kitten videos on Facebook (just kidding).
On the other hand, it is also great to have a shared vocabulary; a list of words I can use freely without having to explain them. Because inferential distance is a thing. (For example, LW allows me to type “inferential distance” without having to explain. Maybe I could just use a hyperlink to the origin of the term. But doing it outside of LW includes a risk of people starting to debate the concept of the “inferential distance” itself, derailing the discussion.) The opposite of public knowledge is the Eternal September.
Maybe “Moloch” is an example that meaningful terms will spread across rationalist websites. (Natural selection of rationalist memes?) Maybe hyperlinking the original source is all it takes; linking to SSC is not more difficult than linking to LW Sequences, or Wikipedia. That is, assuming that the concept is clearly explained in one self-contained article. Which is not always the case.
Consider “motte and bailey”. I consider it a critical rationalist concept, almost as important as “a map is not the territory”. (Technically speaking, it is a narrower version of “a map is not the territory”.) I believe it helps me to see more clearly through most political debates, but it can also be applied outside of politics. And what is the canonical link? Oh, this. So, imagine that I am talking with people who are not regular SSC readers, and we are debating something either unrelated to politics, or at least unrelated to the part of politics that the SSC article talks about, but somehow there appears to be a confusion, which could be easily solved by pointing out that this is yet another instance of the “motte and bailey” fallacy, so I just use these words in a sentence, and provide a hyperlink-explanation to the SSC article. What could possibly go wrong? How could it possibly derail the whole debate?
Okay, maybe the situation with “motte and bailey” could be solved by writing a more neutral article (containing a link to the original article) afterwards, and referring to the neutral article. More generally, maybe we could just maintain a separate Dictionary of Terms Generally Considered Useful by the Rationalist Community. Or maybe the dictionary would suffer the same fate as the Sequences; it would exist, but most new people would completely ignore it, simply because it isn’t standing in the middle of the traffic.
So I guess there needs to be a community which has a community norm of “you must read this information, or else you are not a valid member of this community”. Sounds ugly, when I put it like this, but the opposite is the information just being somewhere without people being able to use it freely in a debate.
No, this:
http://philpapers.org/archive/SHATVO-2.pdf
My problem with the “shared vocabulary” is that as you note yourself here, it implies that something has already been thought through, and it assumes that you have understood the thing properly. So for example if you reject an argument because “that’s an example of a motte and bailey fallacy”, then this only works if it is in fact correct to reject arguments for that reason.
And I don’t think it is correct. One reason why people use a motte and bailey is that they are looking for some common ground with their interlocutor. Take one of Scott’s examples, with this motte and bailey:
God is just the order and love in the universe
God is an extremely powerful supernatural being who punishes my enemies
When the person asserts #1, it is not because they do not believe #2. It is because they are looking for some partial expression of their belief that the other person might accept. In their understanding, the two statements do not contradict one another, even though obviously the second claims a good deal more than the first.
Now Scott says that #1 is “useless,” namely that even if he could theoretically accept the word “God” as applying to this, there is no reason for him to do this, because there is nowhere to go from there. And this might be true. But the fact that #2 is false does not prove that it is true. Most likely, if you work hard, you can find some #3, stronger than #1, but weaker than #2, which will also be defensible.
And it would be right to tell them to do the work that is needed. But it would be wrong to simply say, “Oh, that’s a motte and bailey” and walk away.
This is not merely a criticism of this bit of shared vocabulary, so that it would just be a question of getting the right shared vocabulary. A similar criticism will apply to virtually any possible piece of shared vocabulary—you are always assuming things just by using the vocabulary, and you might be wrong in those assumptions.
Making shared vocabulary common and explicit usually makes it faster to iterate. For example, the EA community converged on the idea of replaceability as an important heuristic for career decisions for a while, and then realized that they’d been putting too much emphasis there and explicitly toned it down. But the general concept had been floating around in discussion space already, giving it a name just made it easier to explicitly think about.
I think I agree with this in one sense and disagree in another. In particular, in regard to “giving it a name just made it easier to explicitly think about” :
I agree that this makes it easier to reason about, and therefore you might come to conclusions faster and so on, even correctly.
I don’t agree that we really made it easier to think about. What we actually did is make it less necessary to think about it at all, in order to come to conclusions. You can see how this works in mathematics, for example. One of the main purpose of the symbols is to abbreviate complicated concepts so that you don’t have to think through them every time they come up.
I think the second point here is also related to my objection in the previous comment. However, overall, the first point might be overall more important, so that the benefit outweighs the costs, especially in terms of benefit to a community.
What are you using this word to mean? At a guess it sounds like, “ideas will float to the surface” but also it does not always mean that, as used in “has percolated”. Percolate relates to filtering of a substance like coffee, to get the good bits from the bad. Can you repeat the above without using this word?
Are we looking to separate and elevate good ideas from the general noise on the interwebs, or are we looking to ensure ideas filter through the diaspora to every little sub group that exists? Or are we looking to filter something else? I am not sure which you are trying to describe.
If you want to describe an earlier post that is well know, and well spread, it should be enough to describe the name of the concept, i.e. crony beliefs. If you want to reference a less well known concept; it should be enough to name the author and link to their post, like if I wanted to refer to the list of common human goals and talk about things that relate to it.
I don’t see the gravity of the problem you are trying to describe with your concerns.