The Ideological Turing Test
Mill states it well: “He who knows only his own side of the case knows little of that.” If someone can correctly explain a position but continue to disagree with it, that position is less likely to be correct. And if ability to correctly explain a position leads almost automatically to agreement with it, that position is more likely to be correct.
[...] Put me and five random liberal social science Ph.D.s in a chat room. Let liberal readers ask questions for an hour, then vote on who isn’t really a liberal. Then put Krugman and five random libertarian social science Ph.D.s in a chat room. Let libertarian readers ask questions for an hour, then vote on who isn’t really a libertarian. Simple as that.
My challenge: Nail down the logistics, and I’ll happily bet money that I fool more voters than Krugman.
Just like Caplan, I’d like to put my money where my mouth is and play in an ideological Turing Test against a Christian blogger.
UPDATE: Two Christians have contacted me to tell me they’re interested. Please suggest format ideas for us to talk over and let me know if you’d like to join in!
update to clarify: When the panel is made up of mostly genuine Blues and one or a few Greens pretending to be Blue, then the judges are all Blue.
- Training Regime Day 17: Deflinching and Lines of Retreat by (21 Apr 2020 17:45 UTC; 24 points)
- *Another* Double Crux Framework by (28 May 2018 23:30 UTC; 15 points)
- 's comment on Categories of Sacredness by (28 Feb 2018 0:43 UTC; 6 points)
- 's comment on The Rubber Hand Illusion and Preaching to the Unconverted by (30 Dec 2014 5:07 UTC; 5 points)
- 's comment on Discussion with Nate Soares on a key alignment difficulty by (12 Apr 2023 18:34 UTC; 3 points)
- How do you know you are right when debating? Calculate your AmIRight score. by (1 Jun 2024 15:55 UTC; 2 points)
A while back, David Brin wrote up a proposal for “disputation arenas”.
An interesting part of the competition/argument process:
This seems like a very fun game, but I’m not sure how much merit it has as a method of determining which side in a debate is the most informed.
Much like the actual Turing Test, it also seems vulnerable to hard-ball inquisitorial tactics. There are probably quite a few questions you could ask which lie outside the ostensible scope of the debate, but which would correlate strongly with a user’s true beliefs, or require faculties the respondent simply doesn’t have in order to answer satisfactorily. I’m pretty sure if I asked a fake-atheist to name ten atheist-related figures that inspired them, they’d come up with a predictable, odd, or obviously Googled list. Equally, if I was posing as a Christian I’d struggle to name ten inspiring Bible passages that weren’t either predictable, odd, or obviously pulled off Google.
I’ve been an atheist for most of my life, and I’m pretty sure I don’t have that many atheism-related figures who’ve inspired me. I’d have a hard time coming up with a list even if I interpreted the question as broadly as possible.
Same here. So if I had to pretend to be a Christian, I’d waffle around a bit just like I do if you ask me that about atheists.
What if it was just “name ten atheists who you find inspiring”?
I’d probably have to do some googling to make sure the list really is composed solely of atheists.
Yes, I think this is the biggest drawback with the proposal. Lots of us understand pretty well the arguments that Christians make for their position, but we wouldn’t be able to produce the shibboleths that Christians would look for.
You could just forbid tactics violating the spirit of the test, such as these ones.
If we take this as a test, It should be noted that it seems to be biased against popular/conventional/establishment/dominant schools of thought. Historically speaking religious minorities often know a lot about the dominant religion, though the slant may be biased.
However depending on which other tests we use in our investigation this might be a feature not a bug.
I think there are several big problems with this that make it a lot less useful than you might hope as a way of telling which side better understands the other side’s arguments—leaving aside the big problem with picking a “side” in an argument. Nonetheless, I think it’s likely to be informative and worth a go!
As stated, it sounds like everyone is expected to be on the same IRC channel. That doesn’t seem like such a good idea—the imitators can for the most part simply listen to the genuines to work out what to say. What you really want is a way to ask the panel a question and have their answers revealed only once they have all been submitted. However, I can’t immediately think of an existing software platform that makes this convenient.
I don’t know how best to minimise the shibboleth problem. At the very least, we could remind panellists and judges that shibboleths may be more easily imitated than real winning arguments. Bear in mind that the judges are effectively all Blues pretending to be Greens, in that they want to test how well the panel understand the arguments against Green positions.
—Granny Weatherwax, Wyrd Sisters (by Terry Pratchett)
In other words, you win this game not by trying to imitate actual people who hold a position, but by imitating the stereotype of people that hold that position.
If the judges were random disinterested people, maybe, but the judges are people sympathetic to the stated position of the panel; they’re less likely to be fooled by someone parroting a stereotype of themselves. I think that to cryonicists, someone trying to fulfill a stereotype of a cryonicists among a panel of genuine cryonicists would stick out like a sore thumb.
Depends on the position. Remember Poe’s Law—there are some positions where it’s impossible to distinguish a parody from the real thing.
There are people who do this for free. They’re usually known as Internet trolls.
No, they can’t fool supporters of their fake position (e.g. trolls on Alternet pretending to be atheists are quite different from trolls who are sincere about atheism and trolling about another topic).
Depends on the troll.
I suspect the results would depend heavily on the jury. Specifically a blue pretending to be a green, could probably fool fellow blues much more easily then he could fool actual greens.
In this proposal, whenever a Blue is pretending to be a Green he is trying to fool Green judges.
Troublesome because you have an uncontrolled variable—the investigative ability of the people you’re trying to fool.
That’s why you make it so that there are a lot of them.
Not noise—I really mean a variable. For example, say you want to tax cigarettes, and someone else wants to ban cigarettes. To find out who knows the other person’s position better you trade places and try to fool some large group of their peers. The taxer chats with some banners for a while, and the banner chats with some taxers for a while. Then the two groups write down how knowledgeable they thought people were.
But what if taxers tend to be introverts, and banners tend to be police officers who are masters at interrogation? Or, more plausibly, what if there are group-specific things that will impact peoples’ judgement of your knowledge other than the substance of the issue? There are all sorts of things that could cause groups to grade transplants differently, and these factors are left uncontrolled if you just do the swap as suggested.
Ah, OK then. (Selection bias.)
If its carreid out it should be entertaining. Some comparisons among other ideological constructs would be interesitng as well. I wonder how say someone like Paul Gottfried or John Derbyshire would do immitating left or Neoconservative positions.
I asked Leah about participating in her test. I think I will find it challenging the emulate a Christian’s viewpoint, instead of mocking it.
That is, normally when I’ve examined Christian thought, I’ve approached it from the pretense “let me see where this fails.”
I’m interested to see if I can construct an argument that I myself as an atheist would find at least superficially compelling—that is, I want to avoid doing what pretty much every popular Christianity book does.
If you try to present radically better arguments for Christianity than you think most Christians present, you will sound out of step with the rest of the panel to an audience of Christian judges.
I think this is correct. If you want to successfully pose as a Christian, you might be well advised to read a bunch of C. S. Lewis, and then imitate his arguments and style. I say this because I think his books constitute the most accessible body of reasonable-but-still-wrong arguments in favour of Christian orthodoxy. If you can quote him, all the better, because being able to quote C. S. Lewis is a high-status marker among people who have both a self-identity as Christians, and a self-identity as intellectuals.
Voting for the Christian / “Christian” half is open—see Leah’s site.
I had fun figuring these out. Once it’s done I’d be interested to see what criteria LW users used to determine someone’s real beliefs...
First round (Christians pretending to be atheists, atheists being sincere) voting is open.
Sounds like a neat idea! I’ve been looking for these kinds of “meta” approaches to political discussion, and this looks like a pretty good one I hadn’t thought of!
An improvement might be to restrict the discussion to a topic narrower than “libertarians vs. liberals” or “christians versus atheists”—for example “labour laws”, “immigration”, “the documentary hypothesis”, “morality of homsexuality”, etc.
This looks like it’d have a much better likelihood-I-change-my-mind / invested-time ratio than an “ordinary” poltical debate.
I asked Leah about participating in her test. I think I will find it challenging the emulate a Christian’s viewpoint, instead of mocking it.
That is, normally when I’ve examined Christian thought, I’ve approached it from the pretense “let me see where this fails.”
I’m interested to see if I can construct an argument that I myself as an atheist would find at least superficially compelling—that is, I want to avoid doing what pretty much every popular Christianity book does.