Thanks for a ton of great tips Anna, just wanted to nit pick on one:
Remember that if reading X-ist books will predictably move your beliefs toward X, and you know there are X-ist books out there, you should move your beliefs toward X already. Remember the Conservation of Expected Evidence more generally.
I suspect that reading enough X-ist books will affect my beliefs for any X (well, nearly any). The key word is enough—I suspect that fully immersing myself in just about any subject, and surround myself entirely by people who advocate it, would significantly alter my beliefs, regardless of the validity of X.
It wouldn’t necessarily make you a believer. Worked example: I joined in the battle of Scientology vs. the Net in 1995 and proceeded to learn a huge amount about Scientology and everything to do with it. I slung the jargon so well that some ex-Scientologists refused to believe I’d never been a member (though I never was). I checked my understanding with ex-Scientologists to see if my understanding was correct, and it largely was.
None of this put me an inch toward joining up. Not even slightly.
To understand something is not to believe it.
That said, it’ll provide a large and detailed pattern in your head for you to form analogies with, good or bad.
I suspect that fully immersing myself in just about any subject, and surround myself entirely by people who advocate it, would significantly alter my beliefs, regardless of the validity of X.
It seems that your experience was learning about anti-Scientology facts while surrounded by people who advocated anti-Scientology.
So it’s completely unsurprising that you remained anti-Scientology.
Had you been learning about Scientology from friends of yours who were Scientologists, you might have had a much harder time maintaining your viewpoint.
Similarly, learning about christianity through the skeptics annotated bible is very different from learning about christianity through a christian youth group.
I actually first started reading alt.religion.scientology because I was interested in the substance of Scientology (SPOILER: there isn’t any) from being a big William S. Burroughs fan. The lunacy is pretty shallow below the surface, which is why the Church was so desperately keen to keep the more esoteric portions from the public eye as long as possible.
But, um, yeah. Point.
OTOH, all the Scientologists I knew personally before that emitted weirdness signals. Thinking back, they behaved like they were trying to live life by a manual rather than by understanding. Memetic cold ahoy!
Interesting! But I do think it’s harder than we imagine to maintain that perfect firewall between arguments you read and arguments you believe (or at least absorb into your decisions). Cases where you’re genuinely uncertain about the truth are probably more salient than cases like Scientology on this front.
Well, yeah. Scientology is sort of the Godwin example of dangerous infectious memes. But I’ve found the lessons most useful in dealing with lesser ones, and it taught me superlative skills in how to inspect memes and logical results in a sandbox.
Perhaps these have gone to the point where I’ve recompartmentalised and need to aggressively decompartmentalise again. Anna Salamon’s original post is IMO entirely too dismissive of the dangers of decompartmentalisation in the Phil Goetz post, which is about people who accidentally decompartmentalise memetic toxic waste and come to the startling realisation they need to bomb academics or kill the infidel or whatever. But you always think it’ll never happen to you. And this is false, because you’re running on unreliable hardware with all manner of exploits and biases, and being able to enumerate them doesn’t grant you immunity. And there are predators out there, evolved to eat people who think it’ll never happen to them.
My own example: I signed up for a multi-level marketing company, which only cost me a year of my life and most of my friends. I should detail precisely how I reasoned myself into it. It was all very logical. The process of reasoning oneself into the mouth of a highly evolved predator tends to be. The cautions my friends and family gave me were all heuristic. This was before I studied Scientology in detail, which would I suspect have given me some immunity.
I should write a post on the subject (see my recent comments) except Anna’s post covers quite a lot of it.
Reading the sucker shoot analogy in a Florence Littauer book (CAUTION: Littauer is memetic toxic waste with some potentially useful bits). That was the last straw after months of doubts, the bit where it went “click! Oh, this is actually really bad for me, isn’t it?” Had my social life been on the internet then (this was 1993) this would have been followed with a “gosh, that was stupid, wasn’t it?” post. I hope.
It may be relevant that I was reading the Littauer book because Littauer’s books and personality theories were officially advocated in the MLM in question (Omegatrend, a schism of Amway) - so it seemed to be coming from inside. I worry slightly that I might have paid insufficient attention had it been from outside.
I’d be interested to know how others (a) suffered a memetic cold (b) got out of it. Possible post material.
Seems to me that, for most questions where there is any real uncertainty, many books are written advocating multiple points of view. If I were to read any one of these books, I would probably move closer to the author’s point of view (since the author will select evidence to support his/her belief), but to know what I would believe after reading all of the books, I would have to actually read them to compare the strength of their arguments.
Yes I think you’re mostly right, but I just don’t think I’m quite good enough to weigh the evidence just right, even when I’m explicitly trying to do. Especially in cases where there is real uncertainty.
If you’re built anything like me, the size of the effect does depend pretty strongly X; some may require a simple book, some may equire a full-fledged immersive indoctrination with a lot of social pressure. So I should move my belief towards any X that sounds like it could convince me with a simple book, which would cover a lot of (conflicting) theories on economics and history, but not a lot of religion or conspiracy theories or nationalist ideologies.
Another belief this would leave me away form is ideas that “people who believe in X are evil/crazy” for a lot of values of X.
Thanks for a ton of great tips Anna, just wanted to nit pick on one:
I suspect that reading enough X-ist books will affect my beliefs for any X (well, nearly any). The key word is enough—I suspect that fully immersing myself in just about any subject, and surround myself entirely by people who advocate it, would significantly alter my beliefs, regardless of the validity of X.
It wouldn’t necessarily make you a believer. Worked example: I joined in the battle of Scientology vs. the Net in 1995 and proceeded to learn a huge amount about Scientology and everything to do with it. I slung the jargon so well that some ex-Scientologists refused to believe I’d never been a member (though I never was). I checked my understanding with ex-Scientologists to see if my understanding was correct, and it largely was.
None of this put me an inch toward joining up. Not even slightly.
To understand something is not to believe it.
That said, it’ll provide a large and detailed pattern in your head for you to form analogies with, good or bad.
Alexflint said:
It seems that your experience was learning about anti-Scientology facts while surrounded by people who advocated anti-Scientology.
So it’s completely unsurprising that you remained anti-Scientology.
Had you been learning about Scientology from friends of yours who were Scientologists, you might have had a much harder time maintaining your viewpoint.
Similarly, learning about christianity through the skeptics annotated bible is very different from learning about christianity through a christian youth group.
I actually first started reading alt.religion.scientology because I was interested in the substance of Scientology (SPOILER: there isn’t any) from being a big William S. Burroughs fan. The lunacy is pretty shallow below the surface, which is why the Church was so desperately keen to keep the more esoteric portions from the public eye as long as possible.
But, um, yeah. Point.
OTOH, all the Scientologists I knew personally before that emitted weirdness signals. Thinking back, they behaved like they were trying to live life by a manual rather than by understanding. Memetic cold ahoy!
Interesting! But I do think it’s harder than we imagine to maintain that perfect firewall between arguments you read and arguments you believe (or at least absorb into your decisions). Cases where you’re genuinely uncertain about the truth are probably more salient than cases like Scientology on this front.
Well, yeah. Scientology is sort of the Godwin example of dangerous infectious memes. But I’ve found the lessons most useful in dealing with lesser ones, and it taught me superlative skills in how to inspect memes and logical results in a sandbox.
Perhaps these have gone to the point where I’ve recompartmentalised and need to aggressively decompartmentalise again. Anna Salamon’s original post is IMO entirely too dismissive of the dangers of decompartmentalisation in the Phil Goetz post, which is about people who accidentally decompartmentalise memetic toxic waste and come to the startling realisation they need to bomb academics or kill the infidel or whatever. But you always think it’ll never happen to you. And this is false, because you’re running on unreliable hardware with all manner of exploits and biases, and being able to enumerate them doesn’t grant you immunity. And there are predators out there, evolved to eat people who think it’ll never happen to them.
My own example: I signed up for a multi-level marketing company, which only cost me a year of my life and most of my friends. I should detail precisely how I reasoned myself into it. It was all very logical. The process of reasoning oneself into the mouth of a highly evolved predator tends to be. The cautions my friends and family gave me were all heuristic. This was before I studied Scientology in detail, which would I suspect have given me some immunity.
I should write a post on the subject (see my recent comments) except Anna’s post covers quite a lot of it.
I hope you’ll also post about how you reasoned yourself out of it.
Reading the sucker shoot analogy in a Florence Littauer book (CAUTION: Littauer is memetic toxic waste with some potentially useful bits). That was the last straw after months of doubts, the bit where it went “click! Oh, this is actually really bad for me, isn’t it?” Had my social life been on the internet then (this was 1993) this would have been followed with a “gosh, that was stupid, wasn’t it?” post. I hope.
It may be relevant that I was reading the Littauer book because Littauer’s books and personality theories were officially advocated in the MLM in question (Omegatrend, a schism of Amway) - so it seemed to be coming from inside. I worry slightly that I might have paid insufficient attention had it been from outside.
I’d be interested to know how others (a) suffered a memetic cold (b) got out of it. Possible post material.
Just re-read this thread, and I’m still keen to hear how you reasoned yourself into it.
I would be interested in reading this, and especially about what caused the initial vulnerability.
Seems to me that, for most questions where there is any real uncertainty, many books are written advocating multiple points of view. If I were to read any one of these books, I would probably move closer to the author’s point of view (since the author will select evidence to support his/her belief), but to know what I would believe after reading all of the books, I would have to actually read them to compare the strength of their arguments.
Yes I think you’re mostly right, but I just don’t think I’m quite good enough to weigh the evidence just right, even when I’m explicitly trying to do. Especially in cases where there is real uncertainty.
If you’re built anything like me, the size of the effect does depend pretty strongly X; some may require a simple book, some may equire a full-fledged immersive indoctrination with a lot of social pressure. So I should move my belief towards any X that sounds like it could convince me with a simple book, which would cover a lot of (conflicting) theories on economics and history, but not a lot of religion or conspiracy theories or nationalist ideologies.
Another belief this would leave me away form is ideas that “people who believe in X are evil/crazy” for a lot of values of X.