The whole idea of having a belief as a litmus test for rationality seems totally backward. The whole point is how you change your beliefs in response to new evidence.
Meanwhile, if a lot of people have a belief that isn’t true it is almost necessarily politically salient. The existence of God isn’t an issue that is debated in the halls of government: but it is still hugely about group identity which means that people can get mind-killed about it. The only reason it works as any kind of litmus test is that everyone here is/was already a part of the same group when it comes to theism.
I think the true objection to Stuart’s post was less about climate change and more about branding Less Wrong with an issue that has ideological salience. And that seems totally fair to me. If you have a one issue litmus test it’s sort of weird to make it one that isn’t specific enough to screen out even the most irrational liberals. At the very least add a sub-test asking if a person thinks carbon emissions are responsible for the Hurricane Sandy disaster, their confidence that climate change causes more hurricanes and what (if any) existential risk they assign to it. Catch the folks who think the moon is made out of gold in the filter.
The whole idea of having a belief as a litmus test for rationality seems totally backward. The whole point is how you change your beliefs in response to new evidence.
I think this is a very uncharitable interpretation of what the post in question is trying to say. First, the post isn’t proposing a litmus test, but a test that is better than theism in identifying irrationality. Second, how would you know if someone changes their beliefs in response to new evidence without assessing their beliefs in relation to shared evidence? There’s no way Stuart was stupid enough to think evidence shouldn’t be shared for this to work.
ETA: I’m not a native speaker, and I’m not sure how people use the word litmus test anymore.
“Litmus test” in common U.S. usage means a quick and treated-as-reliable proxy indicator for whether a system is in a given state. To treat X as a litmus test for rationality, for example, is to be very confident that a system is rational if the system demonstrates X, and (to a lesser extent) to be very confident that a system is irrational if the system fails to demonstrate X.
I kind of want to respond “what hypo rational said,” but let me see if I can say it more clearly:
Yes the point of rationality is how you change your beliefs in response to new evidence, but some beliefs are evidence that the person who holds the belief isn’t doing a very good job of that.
Admittedly, any single belief is just one bit of information about a person’s rationality, and maybe Stuart should have acknowledged that. But it still makes sense to talk about which bits are more informative.
I doubt Stuart meant to suggest AGW should be “the” litmus test for LessWrong, or a central part of LessWrong’s branding, or anything like that. Again, the question is just which bit is more informative.
Yes the point of rationality is how you change your beliefs in response to new evidence, but some beliefs are evidence that the person who holds the belief isn’t doing a very good job of that.
That’s certainly true. I just think you can get a lot more information much faster directly examining how someone’s beliefs change in response to new evidence.
Admittedly, any single belief is just one bit of information about a person’s rationality, and maybe Stuart should have acknowledged that. But it still makes sense to talk about which bits are more informative.
Well, it’s definitely not the bit that isn’t specific enough to provide (much) information about the vast number of people in the world who believe in climate change because it is a tribal signifier. The existence of God is pretty unique in being both insanely improbable and widely believed. Incidentally, Stuart’s post doesn’t actually argue otherwise. His argument actually doesn’t even fit his thesis: what he’s trying to say is that disbelief in anthropogenic climate change is indicative of a higher degree of irrationality than theism, not that it is more indicative. That might actually be true just based on the average denier of climate change but it’s hard to apply that standard universally when the certainty of climate scientists is only at 95%. 5% uncertainty leaves a little room for intelligent, rational skepticism among people who already tend to be suspicious of many established scientific theories. Conversely the median probability assigned to God’s existence in these parts is 0.
In other words: yes, the median climate change denier might indeed by less rational than the median theist. But the probability of anthropogentic climate change being wrong is much higher than the probability that God exists—which makes in unreliable as a test. Also, that’s clearly the quote my opponent will discover if I ever decide to run for public office.
I doubt Stuart meant to suggest AGW should be “the” litmus test for LessWrong, or a central part of LessWrong’s branding, or anything like that. Again, the question is just which bit is more informative.
Eh. Here was his thesis:
Theism is often a default test of irrationality on Less Wrong, but I propose that global warming denial would make a much better candidate.
I sort of feel like the determination that theism is irrational and it’s role as the Plimsoll line for participating at Less Wrong is pretty central to the brand. In a lot of ways the community grew out of the atheist blogosphere and we don’t even really let theists argue here. I know some Right-leaning posters are already leery of a left-ward tilt to Less Wrong: I can imagine them being annoyed by how his proposal sounds.
But at this point I think we’re over-analyzing the post.
I don’t think Stuart’s test is particularly useful by itself, so don’t take this as me defending it. His post is also vague and short enough to allow for several interpretations.
That’s certainly true. I just think you can get a lot more information much faster directly examining how someone’s beliefs change in response to new evidence.
What do you mean by “directly examine”? What if you can’t interact with the person but want to determine whether reading their book is worthwhile for example? Using a few belief litmus tests could be a great way to prevent wasting your time. There are other similar situations.
If there’s anything good about a belief litmus test, it’s that it’s simpler to apply than anything else. Probing someone’s belief structure might take a lot of time, and might be socially unacceptable in certain situations. It might not be easy to assess why a person fails to update, as they might have other conflicting beliefs you’re not aware of. Like any test, there will be false positives and false negatives. I think it’s a matter personal preference how many you’re willing to accept, and depends on how much effort you’re willing to put into testing.
Theism is often a default test of irrationality on Less Wrong, but I propose that global warming denial would make a much better candidate.
A default test, not the default test. I think we’re both nitpicking here and it’s pretty pointless.
I sort of feel like the determination that theism is irrational and it’s role as the Plimsoll line for participating at Less Wrong is pretty central to the brand.
Please define Plimsoll line. Is there a reason you didn’t use a more readily understandable word? I’ve seen theists stepping out of the closet and being upvoted here. It’s just when they come here with the default arguments we’ve seen a million times that they get downvoted to oblivion.
I know some Right-leaning posters are already leery of a left-ward tilt to Less Wrong:
That’s truly bizarre, considering that I basically managed to lose 100 karma points for arguing fairly typical social-democratic positions on LessWrong just yesterday.
Now, yes, “politics is the mind-killer”, but people get mind-killed in a direction, and the direction here is very definitely neoliberal, ie: economically market-populist proprietarian, culturally liberal.
That’s truly bizarre, considering that I basically managed to lose 100 karma points for arguing fairly typical social-democratic positions
Well, one possibility is that fairly typical social-democratic positions are “left” of LW’s earlier position according to those “Right-leaning posters,” and therefore constitute a left-ward tilt from their perspective.
That is entirely possible. However, in that case, I would expect that other people would argue social-democratic positions well (assuming we hold that social-democratic positions have the same prior probability as those of any other ideology of equivalent complexity), and receive upvotes for it. Instead, I just saw an overwhelmingly neoliberal consensus in which I was actually one of the two or three people explaining or advocating left-wing positions at all.
Think of the Talmud’s old heuristic for a criminal court: a clear majority ruling is reliable, but a unanimous or nearly unanimous ruling indicates a failure to consider alternatives.
Now, admittedly, neoliberal positions appear often appealingly simple, even when counterintuitive. The problem is that they appear simple because the complexity is hiding in unexamined assumptions, assumptions often concealed in neat little parables like “money, markets, and businesses arise as a larger-scale elaboration of primitive barter relations”. These parables are simple and sound plausible, so we give them very large priors. Problem is, they are also complete ahistorical, and only sound simple for anthropic reasons (that is: any theory about history which neatly leads to us will sound simpler than one that leads to some alternative present, even if real history was in fact more complicated and our real present less genuinely probable).
So overall, it seems that for LessWrong, any non-neoliberal position (ie: position based on refuting those parables) is going to have a larger inferential distance and take a nasty complexity penalty compared to simply accepting the parables and not going looking for historical evidence. This may be a fault of anthropic bias, or even possibly a fault of Bayesian thinking itself (ie: large priors lead to very-confident belief even in the absence of definite evidence).
Now, admittedly, neoliberal positions appear often appealingly simple, even when counterintuitive. The problem is that they appear simple because the complexity is hiding in unexamined assumptions, assumptions often concealed in neat little parables like “money, markets, and businesses arise as a larger-scale elaboration of primitive barter relations”. These parables are simple and sound plausible, so we give them very large priors. Problem is, they are also complete ahistorical, and only sound simple for anthropic reasons (that is: any theory about history which neatly leads to us will sound simpler than one that leads to some alternative present, even if real history was in fact more complicated and our real present less genuinely probable).
This particular example doesn’t seem troublesome to me, because I’m comfortable with the idea of bartering for debt. That is, my neighbor gives me a cow, and now I owe him one- then I defend his home from raiders, and give him a chicken, and then we’re even. A tinker comes to town, and I trade him a pot of alcohol for a knife because there’s no real trust of future exchanges, and so on. Coinage eventually makes it much easier to keep track of these things, because then we don’t have my neighbor’s subjective estimate of how much I owe him versus my subjective estimate of how much I owe my neighbor, we can count pieces of silver.
Now, suppose I’m explaining to a child how markets work. There are simply less moving pieces to tell it as “twenty chickens for a cow” than “a cow now for something roughly proportional to the value of the cow in the future,” and so that’s the explanation I’ll use, but the theory still works for what actually happened. (Indeed, no doubt you can explain the preference for debt over immediate bartering as having lower frictional costs for transactions.)
In general, it’s important to keep “this is an illustrative example” separate from “this is how it happened,” which I don’t know if various neoliberals have done. Adam Smith, for example, claims that barter would be impractical, and thus people immediately moved to currency, which was sometimes things like cattle but generally something metal.
I would expect that other people would argue social-democratic positions well
In this particular thread or on LW in general?
In the particular thread, it’s likely that such people didn’t have time or inclination to argue, or maybe just missed this whole thing altogether. On LW in general, I don’t know—I haven’t seen enough to form an opinion.
In any case the survey results do not support your thesis that LW is dominated by neoliberals.
but a unanimous or nearly unanimous ruling indicates a failure to consider alternatives.
Haven’t seen much unanimity on sociopolitical issues here.
On the other hand there is that guy Bayes… hmm… what did you say about unanimity? :-D
Problem is, they are also complete ahistorical, and only sound simple for anthropic reasons
Graeber’s views are not quite mainstream consensus ones. And, as you say, *any* historical narrative will sound simple for anthropic reasons—it’s not something specific to neo-liberalism.
Not sure what you are proposing as an alternative to historical narratives leading to what actually happened. Basing theories of reality on counterfactuals doesn’t sound like a good idea to me.
In any case the survey results do not support your thesis that LW is dominated by neoliberals.
The survey results are out? Neat!
Not sure what you are proposing as an alternative to historical narratives leading to what actually happened. Basing theories of reality on counterfactuals doesn’t sound like a good idea to me.
I’m not saying we should base theories on counterfactuals. I’m saying that we should account for anthropic bias when giving out complexity penalties. The real path reality took to produce us is often more complicated than the idealized or imagined path.
Graeber’s view are not quite mainstream consensus ones.
The question is: are they non-mainstream in economics, anthropology, or both? I wouldn’t trust him to make any economic predictions, but if he tells me that the story of barter is false, I’m going to note that his training, employment, and social proof are as an academic anthropologist working with pre-industrial tribal cultures.
At minimum, it does seem like many anthropologists see Graeber’s work as much more tied into his politics than things even often are in that field, and that’s a field that has serious issues with that as a whole.
Considering how many of their comments have been downvoted, including inquiries like this one, and other recent events, such as those discussed by Ialdabaoth and others here, my guess is that’s not what is going on here.
To be clear, I don’t think someone’s net-stalking me. That would be ridiculous. But I do think there’s a certain… tone and voice that’s preferred in a LessWrong post, and I haven’t learned it yet. There’s a way to “sound more rational”, and votes are following that.
While I take your point, it seems unlikely that that’s what’s motivating the response here. eli_sennesh and Eugine_Nier are about as far apart from each other politically as you can get without going into seriously fringe positions, with ialdabaoth in the middle, but there’s evidence of block downvoting for all of them. You’d need a pretty dastardly enemy to explain all of that.
(I don’t think block downvoting’s responsible for most of eli’s recent karma loss, though.)
(I don’t think block downvoting’s responsible for most of eli’s recent karma loss, though.)
Block, meaning organized effort? Definitely not. But I definitely find a −100 karma hit surprising, considering that even very hiveminded places like Reddit are very slow to accumulate comment votes in one direction or the other.
EDIT: And now I’m at +13 karma, which from −48 is simply absurd again. Is the system intended to produce dramatic swings like that? Have I invoked the “complain about downvoting, get upvoted like mad” effect seen normally on Reddit?
There’s a fairly common pattern where someone says something that a small handful of folks downvote, then other folks come along and upvote the comment back to zero because they don’t feel it deserves to be negative, even though they would not have upvoted it otherwise. You’ve been posting a lot lately, so getting shifts of several dozen karma back and forth due to this kind of dynamic is not unheard of, though it’s certainly extreme.
Concerted, not necessarily organized. It’s possible for one person to put a pretty big dent in someone else’s karma if they’re tolerant of boredom and have a reasonable amount of karma of their own; you get four possible downvotes to each upvote of your own (upvotes aren’t capped), which is only rate-limiting if you’re new, downvoting everything you see, or heavily downvoted yourself.
This just happens to have been a sensitive issue recently, as the links in JoshuaZ’s ancestor comment might imply.
I understand block downvoting as a user (one, but possibly more) just going through each and every post by a certain poster and downvoting each one without caring about what it says.
It is not an “organized effort” in the sense of a conspiracy.
Blockvoting may or may not be going on in this case, but at this point, I also assign a high probability that there are people who here downvote essentially all posts that potentially seem to be arguing for positions that are generally seen as to be on the left-end of the political spectrum. That seems include posts which are purely giving data and statistics.
As I mentioned, I accept the block downvoting exists, it’s pretty obvious. However the question is what remains after you filter it out. And as you yourself point out, in this case the remainder is still negative.
I hope you realize the epistemical dangers of automatically considering all negative feedback as malicious machinations of your dastardly enemies...
Of course that would be epistemically dangerous. Dare I say it, as assuming that all language used by people one doesn’t like is adversarial?
More to the point, I haven’t made any such assumption. There are contexts where negative feedback and discussion is genuine and useful, and some of eli’s comments have been unproductive, and I’ve actually downvoted some of them. That doesn’t alter the fact that there’s nothing automatic going on: in the here and now, we have a problem involving at least one person, and likely more, downvoting due primarily for disagreement rather than anything substantial, and that that is coming from a specific end of the political spectrum. That doesn’t say anything about “dastardly enemies”- it simply means that karma results on these specific issues are highly likely in this context to be not representative, especially when people are apparently downvoting Eli’s comments that are literal answers to questions that they don’t like, such as here.
The possibilities that Eli’s comments were downvoted “politically” and that they were downvoted “on merits” are not mutually exclusive. It’s likely that both things happened.
Block down- and up-voting certainly exists. However, as has been pointed out, you should treat this as noise (or, rather, the zero-information “I don’t like you” message) and filter it out to the degree that you can.
Frankly, I haven’t looked carefully at votes in that thread, but some of Eli’s posts were silly enough to downvote on their merits, IMHO. I have a habit of not voting on posts in threads that I participate in, but if I were just an observer, I would have probably downvoted a couple.
The possibilities that Eli’s comments were downvoted “politically” and that they were downvoted “on merits” are not mutually exclusive. It’s likely that both things happened.
I agree that both likely happened. But if a substantial fraction was happening to the first, what does that suggest?
However, as has been pointed out, you should treat this as noise (or, rather, the zero-information “I don’t like you” message) and filter it out to the degree that you can.
Look at short neutral “utility” posts and add back the missing karma to all the rest.
For example if somewhere in the thread there were a post “Could you clarify?” and that post got −2 karma, you would just assume that two people block-downvoted everything and add 2 karma to every post in the thread.
If you want to be more precise about it, you can look at the “% positive” number which will help you figure out how much karma to add back.
For example if somewhere in the thread there were a post “Could you clarify?” and that post got −2 karma, you would just assume that two people block-downvoted everything and add 2 karma to every post in the thread.
So, that seems like a plausible method, and that suggests there’s a −2 to −3 range going on to Eli’s stuff. But that’s a lot of effort, and it means that people reading it or going to get a false feeling of a consensus on LW unless they are aware enough to do that, and moreover, it is, simply put, highly discouraging. Daenery and TimS have both stated that due to this sort of thing (and be clear it is coming disproportionately from a specific end of the political spectrum) that they are less frequently posting on LW. That means that people are actively using the karma system to force a political narrative. Aside from the obvious reasons why that’s bad, that’s also unhelpful if one is actually trying to have discussion that has any decent chance of actually finding out information about reality rather than simply seeing what “side” has won in any given context. I’d rather that LW not turn into the political equivalent of /r/politics on reddit, where despite the nominal goals, certain political opinions drown out almost all dissent. The fact that it would be occurring on the other end of the political spectrum doesn’t help matters. And can easily be particularly damaging given LW’s long-term goals are about rationality, not politics.
For my own trying-to-shut-up part, I do find one thing about “politics is the mind-killer” distinctly weird: the notion that we can seriously discuss morality, ethics, meta-ethics, and Taking Over The World thereby, and somehow expect never to arrive at a matter of political controversy.
For one example, an FAI would likely severely reduce the resource-income and social status of every single currently-active politician, left or right, up or down.
For another, more difficult, example, I can’t actually think of how you would do, say, CEV without some kind of voting and weighting system over the particular varieties of human values. Once you’ve got some notion of having to measure the values of at least a representative sample of everyone in the world and extrapolate those, you are innately in “political” territory. Once you need to talk of resource tradeoffs between values, you are innately in “economic” territory. Waving your arms and saying, “Friendly Superintelligence!” won’t actually tell us anything about what algorithm that thing is actually running.
If I may don my Evil Hansonian hat for a moment, conventional politics isn’t so much about charting the future of our society as about negotiating the power relationships between tribal alignments. Values and ethical preferences and vague feelings of ickiness go into those alignments (and then proceed to feed back out of them), but it’s far rarer for people to support political factions out of de-novo ethical reasoning than you’d guess from talking to them about it. The mind-killer meme is fundamentally an encouragement to be mindful of that, especially of the nasty ideological feedback loops that it tends to imply, and a suggestion to focus on object-level issues where the feedback isn’t quite so intense.
One consequence of this is that political shifts happen at or above human timescales, as their subjects become things that established tribes notice they can fight over. If you happen to be a singularitarian, then, you probably believe that the kinds of technological and social changes that LW talks about will at some point—probably soon, possibly already—be moving faster than politics can keep up with. Speaking for myself, I expect anything that conventional legislatures or political parties say about AI to matter about as much as the RIAA did when they went after Napster, and still less once we’re in a position to be talking seriously about strong, friendly artificial intelligence.
More importantly from our perspective, though, anything conventional politics doesn’t care about yet is also something that we have a considerably better chance of talking about sanely. We may be—in fact, we’re certainly—in the territory of politics in the sense of subjects relevant to the future of the polis, but as long as identity considerations and politics-specific “conventional wisdom” stay relatively distant from our reasoning, we can expect our minds to remain relatively happy and unkilled.
Yeah, this comes up from time to time. My own approach to it is to (attempt as best as I can to) address the underlying policy question while avoiding language that gets associated with particular partisan groups.
For example, I might discuss how a Blue politician might oppose FAI because they value their social status, or how a Green politician might expect a Blue politician to oppose FAI for such a reason even though the Green politician is not driven purely by such motives, or whatever… rather than using other word-pairs like (Republican/Democrat), (liberal/conservative), (reactionary/progressive), or whatever.
If I get to a point in that conversation where the general points are clearly understood, and to make further progress I need to actually get into specifics about specific politicians and political parties, well, OK, I decide what to do when that happens. But that’s not where I start.
And I agree that the CEV version of that conversation is more difficult, and needs to be approached with more care to avoid being derailed by largely irrelevant partisan considerations, and that the same is true more generally about specific questions related to value tradeoffs and, even more generally, questions about where human values conflict with one another in the first place.
I don’t think the usual mantra of politics is the mind-killer is meant to avoid all political issues, although that would be one interpretation. Rather, there are two distinct observations: one is purely descriptive, that politics can be a mind-killer. The second is proscriptive- to possibly refrain when possible from discussing politics until our general level of rationality improves. Unfortunately, that’s fairly difficult, because many of these issues matter. Moreover, it connects with certain problems where counter-intuitive or contrarian ideas are seen as somehow less political than more mainstream ones.
Moreover, it connects with certain problems where counter-intuitive or contrarian ideas are seen as somehow less political than more mainstream ones.
That’s… an excellent way of putting it. Non-mainstream political “tribes” are considered “less political” precisely because they don’t stand any chance of actually winning elections in the real world, so they get a Meta-Contrarian Boost on the internet. The usual ones I see are anarchists, libertarians, and neo-reactionaries.
Empirically I don’t think this is true. Minority political tribes sometimes get a pass for organizing themselves around things that aren’t partisan issues, or are only minor partisan issues, in the mainstream—the Greens sometimes benefit from this in US discourse, although they’re a complicated and very regionally dependent case—but as soon as you stake out a position on a mainstream claim, even if your reasoning is very different from the norm, you should expect to be attacked as viciously as any mainstream wonk. I expect neoreaction, for example, would have met with a much less heated reception if it weren’t for its views on race.
Minority views do get a boost on the Internet, but I think that has more to do with the echo-chamber effects that it encourages. It’s far easier to find or collect a group of people that all agree with you on Reddit or Tumblr than it is out there in the slow, short-range world of blood and bone.
That’s… an excellent way of putting it. Non-mainstream political “tribes” are considered “less political” precisely because they don’t stand any chance of actually winning elections in the real world, so they get a Meta-Contrarian Boost on the internet. The usual ones I see are anarchists, libertarians, and neo-reactionaries.
Considered where, and by whom? Because that is completely unlike my experience. On the Usenet groups rec.arts.sf.*, it was (I have not read Usenet for many years) absolutely standard that Progressive ideas were seen as non-political, while the merest hint of disagreement would immediately be piled on as “introducing politics to the discussion”. And the reactosphere is intensely aware that what they are talking is politics.
I generally agree with this post, but since people’s beliefs are evidence for how they change their beliefs in response to evidence, I would call it bias-inducing and usually tribal cheering instead of totally backwards.
If not “totally backwards” surely “orthogonal”. Why not a test that supplies it’s own evidence and asks the one being tested to come to a conclusion? Like the Amanda Knox case was for people here who hadn’t heard of it before reading about it here.
I wouldn’t call it orthogonal either. Rationality is about having correct beliefs, and I would label a belief-based litmus test rational to the extent it’s correct.
Writing a post about how $political_belief is a litmus test is probably a bad idea because of the reasons you mentioned.
Rationality is about have correct beliefs. But a single belief that has only two possible answers is never going to stand in for the entirety of a person’s belief structure. That’s why you have to look at the process by which a person forms beliefs to have any idea if they are rational.
I’d call it the “Paying-Good-Attention-While-Doing-Simple-Math Test”. :D
But yeah… I can imagine that something similarly simple could be an important part of rationality. Some simple task that predicts the ability to do more complex tasks of a similar type.
However, in that case the test will resemble a kind of puzzle, instead of pattern-matching “Do you agree with Greens?”
Specifically for updating, I can imagine a test where the person is gradually given more and more information; the initial information is an evidence of an outcome “A”, but most of the latter information is an evidence of an outcome “B”. The person is informally asked to make a guess soon after the beginning (when the reasonable answer is “A”), and at the end they are asked to provide a final answer. Some people would probably get stuck as “A”, and some would update to “B”. But the test would involve some small numbers, shapes, coins, etc.; not real-life examples.
Specifically for updating, I can imagine a test where the person is gradually given more and more information; the initial information is an evidence of an outcome “A”, but most of the latter information is an evidence of an outcome “B”. The person is informally asked to make a guess soon after the beginning (when the reasonable answer is “A”), and at the end they are asked to provide a final answer. Some people would probably get stuck as “A”, and some would update to “B”. But the test would involve some small numbers, shapes, coins, etc.; not real-life examples.
I’ve seen experiments that tested this; I thought they were mentioned in Thinking and Deciding or Thinking Fast and Slow, but I didn’t see it in a quick check of either of those. If I recall the experimental setup correctly (I doubt I got the numbers right), they began with a sequence that was 80% red and 20% blue, which switched to being 80% blue and 20% red after n draws. The subjects’ estimate that the next draw would be red stayed above 50% for significantly longer than n draws from the second distribution, and some took until 2n or 3n draws from the second distribution to assign 50% chance to each, at which point almost two thirds of the examples they had seen were blue!
But the test would involve some small numbers, shapes, coins, etc.; not real-life examples.
I dunno… people who do fine at the Wason selection task with ages and drinks get it wrong with numbers and colours. (I’m not sure whether that’s a bug or a feature.)
That seems to me like a reason not to test the skill on real-life examples.
We wouldn’t want a rationality test that a person can pass with original wording, but will fail if we replace “Republicans” by “Democrats”… or by Green aliens. We wouldn’t want the person to merely recognize logical fallacies when spoken by Republicans. This is in my opinion a risk with real-life examples. Is the example with drinking age easier because it is easier to imagine, or because it is something we already agree with?
Okay, I am curious here… what exactly would happen if we replaced the Wason selection task with something that uses words from real life (is less abstract), but is not an actual rule (therefore it cannot be answered using only previous experience)? For example: “Only dogs are allowed at jumping competitions, cats are not allowed. We have a) a dog going to unknown competition; b) a cat going to unknown competition; c) an unknown animal going to swimming competition, and d) an unknown animal going to jumping competition—which of these cases do you have to check thoroughly to make sure the rule is not broken?”
I generally agree with this post, but since people’s beliefs are evidence for how they change their beliefs in response to evidence, I would call it bias-inducing and usually tribal cheering instead of totally backwards.
If I would want to estimate people rationality from beliefs I would look at whether the belief is nuanced. There are a lot of people who say irrational stuff such that they evidence we have for global warming is comparable to the evidence we have for evolution. In reality the p value doesn’t even approach the 5 sigma level that you need to validate a result about a new result in particle physics.
It’s just as irrational as being a global warming denier who thinks that p(global warming)<0.5.
Yet we do see smart people making both mistakes. You have smart people who claim that the evidence for global warming is comparable to evolution and you have smart people who are global warming deniers.
People don’t get mind killed by political issues because they are dumb. It might be completely rational for them because signaling is more important for them. If you want a useful metric do judge someone rationality don’t take something where group identities matter a good deal.
The metric is just too noisy because the person might get something from signaling group identity.
I think the only reason to choose such a metric is because you get yourself mindkilled and want to label people who don’t belong to your tribe as irrational and seek some rationalisation for it.
As far as empirics go, collegue educated Republicans just have a higher rate of climate change denial than Republicans who didn’t go to collegue.
While we can discuss whether collegue causes people to be more rational it certainly correlates with it.
If you want to use beliefs to judge people rationality, calibrate the test. Give people ratioanlity quizes and quiz them for their beliefs. If you get strong correlations you have something that you can use.
Don’t intellectually analyse the content of the beliefs and think about what rational people should believe if you want an effective metric.
RETRACTED: It wasn’t my intention to start another global warming debate.
If I would want to estimate people rationality from beliefs I would look at whether the belief is nuanced.
Lots of insane beliefs are nuanced.
In reality the p value doesn’t even approach the 5 sigma level that you need to validate a result about a new result in particle physics.
Requiring the same strength of evidence from climate science as from particle physics would be insane.
There are a lot of people who say irrational stuff such that they evidence we have for global warming is comparable to the evidence we have for evolution.
From Stuart’s post: “Of course, reverse stupidity isn’t intelligence: simply because one accepts AGW, doesn’t make one more rational.”
People don’t get mind killed by political issues because they are dumb. It might be completely rational for them because signaling is more important for them.
Choosing to signal wouldn’t be mindkill as it’s understood hjhink the only reason to choose such a metric is because you get yourself mindkilled and want to label people who don’t belong to your tribe as irrational and seek some rationalisation for it.
Labeling people seems to be exactly what you’re doing yourself here. I can think of at least three more reasons.
I think Stuart simply underestimated the local mindkill caused by global warming debate in other people, or failed to understand that local mindkill isn’t necessarily a good metric for irrationality. Neither of those require him to be mindkilled about the topic himself. One possibility is he failed to evaluate evidence of global warming himself and overestimated the probability of the relevant propositions.
You seem to be conflating intelligence and rationality in this comment. You probably know they’re not the same thing.
All this being said, I don’t agree with what Stuart was saying in his post. I have no opinion of global warming and haven’t read about it much.
Requiring the same strength of evidence from climate science as from particle physics would be insane.
What do you mean with “require”? If I say that climate science has the same strength of evidence as evolution than we can debate whether climate change does fulfill the 5 sigma criteria criteria.
I think it does, therefore the strength of evidence for climate change is not the same as the strength of evidence for evolution.
Why does it matter? It a X-risk that global warming doesn’t really exist and we do geoengineering that seriously wrecks our planet.
That risk might be something like p=0.001 but it does exist. It’s greater than the risk of an asteroid destroying our civilisation in the next 100 years.
To the extend that one cares about X-risks it’s important to distinguish claims with 2-3 sigma from those who pass 5 sigmas. It’s just not the same level of evidence.
If we want to stay alive over the next hundred years it’s important that decision makers in our society don’t manuver us into an X-risk because they treat 2-3 sigma the same way as they treat treat 5 sigmas.
You seem to be conflating intelligence and rationality in this comment.
I don’t use the word intelligence in the comment you quote. I use it in another post as a proxy variable.
I equate rationality for the ability to update your beliefs in order to win.
You used the words smart and dumb, I suppose that counts. I failed to understand most of your reply.
What do you mean with “require”?
I mean you don’t need to be even nearly that certain for the findings to be actionable.
It a X-risk that global warming doesn’t really exist and we do geoengineering
What’s the expected utility of that compared to the expected utility of AGW? If you’re too uncertain, why not just try to drastically reduce emissions instead of do major geoengineering? What’s the expected utility of reducing emissions?
I mean you don’t need to be even nearly that certain for the findings to be actionable.
If I ask “What’s the evidence for global warming being real?” in searching for an accurate description of the world. Having accurate maps of the world is useful.
In the above example, saying that the evidence for global warming is like that for evolution is like claiming the moon is made of cheese.
The belief might help you to convince people to reduce emissions. Believing that the moon is made of cheese might help you to discourage people from going to the moon.
If the reason that someone advocates the ridiculous claim that the evidence for global warming is comparable to that for evolution, is that it helps him convince people to lower emission that person is mindkilled by his politics.
What’s the expected utility of that compared to the expected utility of AGW? If you’re too uncertain, why not just try to drastically reduce emissions instead of do major geoengineering? What’s the expected utility of reducing emissions?
Right, because our political leaders excel at doing rational good expected utility comparisions…
Memes exist in the real world. They have effects. Promoting false beliefs about the certainity of science has dangers.
I’m not in the position to have the power to choose that the world drastically reduces emissions or whether it does major geoengineering and scientists aren’t either. Scientists do have a social responsiblity to promote accurate beliefs about the world.
Whether or not we should reduce emissions is a different question. If you can’t mentally separate: “Should we reduce emissions” from “What’s the evidence for global warming?” you likely mindkilled about the second question and hold beliefs that aren’t accurate descriptions of reality.
What’s the expected utility of that compared to the expected utility of AGW? If you’re too uncertain, why not just try to drastically reduce emissions instead of do major geoengineering? What’s the expected utility of reducing emissions?
The current understanding of climate sensitivity is that since Carbon Dioxide gas will remain in the upper atmosphere for decades (and possibly centuries) even a complete halt on emissions will not avert warming predicted for the next century or so. And the models currently favored have pretty dire predictions for that level of warming, even if they’re less severe than the alternative.
The only realistic solution, and naturally the one most strongly opposed by environmental groups, is solar radiation management. This would be very expensive, about $700M a year according to David Keith, and has potential risks which should be tested before any implementation plan. So not a silver bullet, but still much cheaper and safer in the long run than the standard environmental agenda even according to their own data.
(Note: I am assuming for the sake of argument that current climate models are accurate, but that is an assumption which should be questioned. Climate modeling is still in it’s infancy and most existing models have difficulty with predictions even as close as a decade out. Warming is probably happening but that does not mean that any given prediction of warming is accurate, for reasons which should be obvious.)
The current understanding of climate sensitivity is that since Carbon Dioxide gas will remain in the upper atmosphere for decades (and possibly centuries) even a complete halt on emissions will not avert warming predicted for the next century or so.
Methane has a shorter lifetime, though (though my five minutes’ research tells me we’ve already stopped increasing methane emissions).
Are you saying that solar radiation management is an alternative to long-term emissions reduction? Or that, in addition to eventually tapering off greenhouse gas emissions, we’re going to have to do something to keep temperatures down, and the best option is solar radiation management?
(edit: apparently I wrote social radiation management)
Reducing emissions is a good goal, but energy needs will continue to increase even as we decrease the number of tons of carbon dioxide per kWh. As the population increases and becomes more wealthy there’s not much we can do but put out more carbon dioxide; that’s one of the reasons people bent on lowering world population and wealth have attached themselves to the environmental movement.
If the stigma against nuclear power goes away, or the technological issues which make speculative energy sources like wind/solar/fusion unprofitable are resolved, we could see a bigger dip but even then the century-long trend will probably be one of increase. SRM is the most realistic way I can think of to head off serious disasters until then.
The whole point is how you change your beliefs in response to new evidence.
Of course the general concept of using a belief as a litmus test for rationality is foolish. But frankly, it’s not possible at this point to have not been introduced to evidence about human-caused global warming. The people to which this test would be applied have been introduced to this new evidence and already failed to update.
And if someone lives in such a secluded bubble of information that they are truly getting information that would lead a rationalist to decry AGW, I think it safe to say that that person is probably not a rationalist. Someone in such a bubble would have no impetus to become a rationalist in the first place.
The whole idea of having a belief as a litmus test for rationality seems totally backward. The whole point is how you change your beliefs in response to new evidence.
Meanwhile, if a lot of people have a belief that isn’t true it is almost necessarily politically salient. The existence of God isn’t an issue that is debated in the halls of government: but it is still hugely about group identity which means that people can get mind-killed about it. The only reason it works as any kind of litmus test is that everyone here is/was already a part of the same group when it comes to theism.
I think the true objection to Stuart’s post was less about climate change and more about branding Less Wrong with an issue that has ideological salience. And that seems totally fair to me. If you have a one issue litmus test it’s sort of weird to make it one that isn’t specific enough to screen out even the most irrational liberals. At the very least add a sub-test asking if a person thinks carbon emissions are responsible for the Hurricane Sandy disaster, their confidence that climate change causes more hurricanes and what (if any) existential risk they assign to it. Catch the folks who think the moon is made out of gold in the filter.
I think this is a very uncharitable interpretation of what the post in question is trying to say. First, the post isn’t proposing a litmus test, but a test that is better than theism in identifying irrationality. Second, how would you know if someone changes their beliefs in response to new evidence without assessing their beliefs in relation to shared evidence? There’s no way Stuart was stupid enough to think evidence shouldn’t be shared for this to work.
ETA: I’m not a native speaker, and I’m not sure how people use the word litmus test anymore.
“Litmus test” in common U.S. usage means a quick and treated-as-reliable proxy indicator for whether a system is in a given state. To treat X as a litmus test for rationality, for example, is to be very confident that a system is rational if the system demonstrates X, and (to a lesser extent) to be very confident that a system is irrational if the system fails to demonstrate X.
This is how I meant it.
That’s what I thought first too, but it seems to also have a political meaning.
You mean the test can be completely unreliable, like many political litmus tests probably are?
Yes, I do mean that.
What a sadly disfigured figure of speech. Chemists would disapprove :(
I wonder if there are many more like it.
That’s pretty much the same meaning; just read “person or policy” for “system”, and “ideologically acceptable” for “in a given state”.
The main difference is the test doesn’t have to be any good.
I kind of want to respond “what hypo rational said,” but let me see if I can say it more clearly:
Yes the point of rationality is how you change your beliefs in response to new evidence, but some beliefs are evidence that the person who holds the belief isn’t doing a very good job of that.
Admittedly, any single belief is just one bit of information about a person’s rationality, and maybe Stuart should have acknowledged that. But it still makes sense to talk about which bits are more informative.
I doubt Stuart meant to suggest AGW should be “the” litmus test for LessWrong, or a central part of LessWrong’s branding, or anything like that. Again, the question is just which bit is more informative.
That’s certainly true. I just think you can get a lot more information much faster directly examining how someone’s beliefs change in response to new evidence.
Well, it’s definitely not the bit that isn’t specific enough to provide (much) information about the vast number of people in the world who believe in climate change because it is a tribal signifier. The existence of God is pretty unique in being both insanely improbable and widely believed. Incidentally, Stuart’s post doesn’t actually argue otherwise. His argument actually doesn’t even fit his thesis: what he’s trying to say is that disbelief in anthropogenic climate change is indicative of a higher degree of irrationality than theism, not that it is more indicative. That might actually be true just based on the average denier of climate change but it’s hard to apply that standard universally when the certainty of climate scientists is only at 95%. 5% uncertainty leaves a little room for intelligent, rational skepticism among people who already tend to be suspicious of many established scientific theories. Conversely the median probability assigned to God’s existence in these parts is 0.
In other words: yes, the median climate change denier might indeed by less rational than the median theist. But the probability of anthropogentic climate change being wrong is much higher than the probability that God exists—which makes in unreliable as a test. Also, that’s clearly the quote my opponent will discover if I ever decide to run for public office.
Eh. Here was his thesis:
I sort of feel like the determination that theism is irrational and it’s role as the Plimsoll line for participating at Less Wrong is pretty central to the brand. In a lot of ways the community grew out of the atheist blogosphere and we don’t even really let theists argue here. I know some Right-leaning posters are already leery of a left-ward tilt to Less Wrong: I can imagine them being annoyed by how his proposal sounds.
But at this point I think we’re over-analyzing the post.
I don’t think Stuart’s test is particularly useful by itself, so don’t take this as me defending it. His post is also vague and short enough to allow for several interpretations.
What do you mean by “directly examine”? What if you can’t interact with the person but want to determine whether reading their book is worthwhile for example? Using a few belief litmus tests could be a great way to prevent wasting your time. There are other similar situations.
If there’s anything good about a belief litmus test, it’s that it’s simpler to apply than anything else. Probing someone’s belief structure might take a lot of time, and might be socially unacceptable in certain situations. It might not be easy to assess why a person fails to update, as they might have other conflicting beliefs you’re not aware of. Like any test, there will be false positives and false negatives. I think it’s a matter personal preference how many you’re willing to accept, and depends on how much effort you’re willing to put into testing.
A default test, not the default test. I think we’re both nitpicking here and it’s pretty pointless.
Please define Plimsoll line. Is there a reason you didn’t use a more readily understandable word? I’ve seen theists stepping out of the closet and being upvoted here. It’s just when they come here with the default arguments we’ve seen a million times that they get downvoted to oblivion.
That’s truly bizarre, considering that I basically managed to lose 100 karma points for arguing fairly typical social-democratic positions on LessWrong just yesterday.
Now, yes, “politics is the mind-killer”, but people get mind-killed in a direction, and the direction here is very definitely neoliberal, ie: economically market-populist proprietarian, culturally liberal.
Well, one possibility is that fairly typical social-democratic positions are “left” of LW’s earlier position according to those “Right-leaning posters,” and therefore constitute a left-ward tilt from their perspective.
Have you considered that you lost your karma not because you argued typical social-democratic positions, but because you argued them badly?
That is entirely possible. However, in that case, I would expect that other people would argue social-democratic positions well (assuming we hold that social-democratic positions have the same prior probability as those of any other ideology of equivalent complexity), and receive upvotes for it. Instead, I just saw an overwhelmingly neoliberal consensus in which I was actually one of the two or three people explaining or advocating left-wing positions at all.
Think of the Talmud’s old heuristic for a criminal court: a clear majority ruling is reliable, but a unanimous or nearly unanimous ruling indicates a failure to consider alternatives.
Now, admittedly, neoliberal positions appear often appealingly simple, even when counterintuitive. The problem is that they appear simple because the complexity is hiding in unexamined assumptions, assumptions often concealed in neat little parables like “money, markets, and businesses arise as a larger-scale elaboration of primitive barter relations”. These parables are simple and sound plausible, so we give them very large priors. Problem is, they are also complete ahistorical, and only sound simple for anthropic reasons (that is: any theory about history which neatly leads to us will sound simpler than one that leads to some alternative present, even if real history was in fact more complicated and our real present less genuinely probable).
So overall, it seems that for LessWrong, any non-neoliberal position (ie: position based on refuting those parables) is going to have a larger inferential distance and take a nasty complexity penalty compared to simply accepting the parables and not going looking for historical evidence. This may be a fault of anthropic bias, or even possibly a fault of Bayesian thinking itself (ie: large priors lead to very-confident belief even in the absence of definite evidence).
This particular example doesn’t seem troublesome to me, because I’m comfortable with the idea of bartering for debt. That is, my neighbor gives me a cow, and now I owe him one- then I defend his home from raiders, and give him a chicken, and then we’re even. A tinker comes to town, and I trade him a pot of alcohol for a knife because there’s no real trust of future exchanges, and so on. Coinage eventually makes it much easier to keep track of these things, because then we don’t have my neighbor’s subjective estimate of how much I owe him versus my subjective estimate of how much I owe my neighbor, we can count pieces of silver.
Now, suppose I’m explaining to a child how markets work. There are simply less moving pieces to tell it as “twenty chickens for a cow” than “a cow now for something roughly proportional to the value of the cow in the future,” and so that’s the explanation I’ll use, but the theory still works for what actually happened. (Indeed, no doubt you can explain the preference for debt over immediate bartering as having lower frictional costs for transactions.)
In general, it’s important to keep “this is an illustrative example” separate from “this is how it happened,” which I don’t know if various neoliberals have done. Adam Smith, for example, claims that barter would be impractical, and thus people immediately moved to currency, which was sometimes things like cattle but generally something metal.
In this particular thread or on LW in general?
In the particular thread, it’s likely that such people didn’t have time or inclination to argue, or maybe just missed this whole thing altogether. On LW in general, I don’t know—I haven’t seen enough to form an opinion.
In any case the survey results do not support your thesis that LW is dominated by neoliberals.
Haven’t seen much unanimity on sociopolitical issues here.
On the other hand there is that guy Bayes… hmm… what did you say about unanimity? :-D
Graeber’s views are not quite mainstream consensus ones. And, as you say, *any* historical narrative will sound simple for anthropic reasons—it’s not something specific to neo-liberalism.
Not sure what you are proposing as an alternative to historical narratives leading to what actually happened. Basing theories of reality on counterfactuals doesn’t sound like a good idea to me.
The survey results are out? Neat!
I’m not saying we should base theories on counterfactuals. I’m saying that we should account for anthropic bias when giving out complexity penalties. The real path reality took to produce us is often more complicated than the idealized or imagined path.
The question is: are they non-mainstream in economics, anthropology, or both? I wouldn’t trust him to make any economic predictions, but if he tells me that the story of barter is false, I’m going to note that his training, employment, and social proof are as an academic anthropologist working with pre-industrial tribal cultures.
Previous years’ survey results: 2012, 2011, 2009. The 2013 survey is currently ongoing.
How would that work?
I am not sure what the mainstream consensus in anthropology looks like, but I have the impression that Graeber’s research is quite controversial.
At minimum, it does seem like many anthropologists see Graeber’s work as much more tied into his politics than things even often are in that field, and that’s a field that has serious issues with that as a whole.
Considering how many of their comments have been downvoted, including inquiries like this one, and other recent events, such as those discussed by Ialdabaoth and others here, my guess is that’s not what is going on here.
To be clear, I don’t think someone’s net-stalking me. That would be ridiculous. But I do think there’s a certain… tone and voice that’s preferred in a LessWrong post, and I haven’t learned it yet. There’s a way to “sound more rational”, and votes are following that.
I hope you realize the epistemical dangers of automatically considering all negative feedback as malicious machinations of your dastardly enemies...
While I take your point, it seems unlikely that that’s what’s motivating the response here. eli_sennesh and Eugine_Nier are about as far apart from each other politically as you can get without going into seriously fringe positions, with ialdabaoth in the middle, but there’s evidence of block downvoting for all of them. You’d need a pretty dastardly enemy to explain all of that.
(I don’t think block downvoting’s responsible for most of eli’s recent karma loss, though.)
Block, meaning organized effort? Definitely not. But I definitely find a −100 karma hit surprising, considering that even very hiveminded places like Reddit are very slow to accumulate comment votes in one direction or the other.
EDIT: And now I’m at +13 karma, which from −48 is simply absurd again. Is the system intended to produce dramatic swings like that? Have I invoked the “complain about downvoting, get upvoted like mad” effect seen normally on Reddit?
There’s a fairly common pattern where someone says something that a small handful of folks downvote, then other folks come along and upvote the comment back to zero because they don’t feel it deserves to be negative, even though they would not have upvoted it otherwise. You’ve been posting a lot lately, so getting shifts of several dozen karma back and forth due to this kind of dynamic is not unheard of, though it’s certainly extreme.
Concerted, not necessarily organized. It’s possible for one person to put a pretty big dent in someone else’s karma if they’re tolerant of boredom and have a reasonable amount of karma of their own; you get four possible downvotes to each upvote of your own (upvotes aren’t capped), which is only rate-limiting if you’re new, downvoting everything you see, or heavily downvoted yourself.
This just happens to have been a sensitive issue recently, as the links in JoshuaZ’s ancestor comment might imply.
Well, I’m sorry for kvetching, then.
I understand block downvoting as a user (one, but possibly more) just going through each and every post by a certain poster and downvoting each one without caring about what it says.
It is not an “organized effort” in the sense of a conspiracy.
Blockvoting may or may not be going on in this case, but at this point, I also assign a high probability that there are people who here downvote essentially all posts that potentially seem to be arguing for positions that are generally seen as to be on the left-end of the political spectrum. That seems include posts which are purely giving data and statistics.
Ah, well. I blame Clippy, then.
As I mentioned, I accept the block downvoting exists, it’s pretty obvious. However the question is what remains after you filter it out. And as you yourself point out, in this case the remainder is still negative.
Of course that would be epistemically dangerous. Dare I say it, as assuming that all language used by people one doesn’t like is adversarial?
More to the point, I haven’t made any such assumption. There are contexts where negative feedback and discussion is genuine and useful, and some of eli’s comments have been unproductive, and I’ve actually downvoted some of them. That doesn’t alter the fact that there’s nothing automatic going on: in the here and now, we have a problem involving at least one person, and likely more, downvoting due primarily for disagreement rather than anything substantial, and that that is coming from a specific end of the political spectrum. That doesn’t say anything about “dastardly enemies”- it simply means that karma results on these specific issues are highly likely in this context to be not representative, especially when people are apparently downvoting Eli’s comments that are literal answers to questions that they don’t like, such as here.
The possibilities that Eli’s comments were downvoted “politically” and that they were downvoted “on merits” are not mutually exclusive. It’s likely that both things happened.
Block down- and up-voting certainly exists. However, as has been pointed out, you should treat this as noise (or, rather, the zero-information “I don’t like you” message) and filter it out to the degree that you can.
Frankly, I haven’t looked carefully at votes in that thread, but some of Eli’s posts were silly enough to downvote on their merits, IMHO. I have a habit of not voting on posts in threads that I participate in, but if I were just an observer, I would have probably downvoted a couple.
I agree that both likely happened. But if a substantial fraction was happening to the first, what does that suggest?
And how do you suggest one do so in this context?
Look at short neutral “utility” posts and add back the missing karma to all the rest.
For example if somewhere in the thread there were a post “Could you clarify?” and that post got −2 karma, you would just assume that two people block-downvoted everything and add 2 karma to every post in the thread.
If you want to be more precise about it, you can look at the “% positive” number which will help you figure out how much karma to add back.
I am not sure it’s worth the bother, though.
So, that seems like a plausible method, and that suggests there’s a −2 to −3 range going on to Eli’s stuff. But that’s a lot of effort, and it means that people reading it or going to get a false feeling of a consensus on LW unless they are aware enough to do that, and moreover, it is, simply put, highly discouraging. Daenery and TimS have both stated that due to this sort of thing (and be clear it is coming disproportionately from a specific end of the political spectrum) that they are less frequently posting on LW. That means that people are actively using the karma system to force a political narrative. Aside from the obvious reasons why that’s bad, that’s also unhelpful if one is actually trying to have discussion that has any decent chance of actually finding out information about reality rather than simply seeing what “side” has won in any given context. I’d rather that LW not turn into the political equivalent of /r/politics on reddit, where despite the nominal goals, certain political opinions drown out almost all dissent. The fact that it would be occurring on the other end of the political spectrum doesn’t help matters. And can easily be particularly damaging given LW’s long-term goals are about rationality, not politics.
For my own trying-to-shut-up part, I do find one thing about “politics is the mind-killer” distinctly weird: the notion that we can seriously discuss morality, ethics, meta-ethics, and Taking Over The World thereby, and somehow expect never to arrive at a matter of political controversy.
For one example, an FAI would likely severely reduce the resource-income and social status of every single currently-active politician, left or right, up or down.
For another, more difficult, example, I can’t actually think of how you would do, say, CEV without some kind of voting and weighting system over the particular varieties of human values. Once you’ve got some notion of having to measure the values of at least a representative sample of everyone in the world and extrapolate those, you are innately in “political” territory. Once you need to talk of resource tradeoffs between values, you are innately in “economic” territory. Waving your arms and saying, “Friendly Superintelligence!” won’t actually tell us anything about what algorithm that thing is actually running.
If I may don my Evil Hansonian hat for a moment, conventional politics isn’t so much about charting the future of our society as about negotiating the power relationships between tribal alignments. Values and ethical preferences and vague feelings of ickiness go into those alignments (and then proceed to feed back out of them), but it’s far rarer for people to support political factions out of de-novo ethical reasoning than you’d guess from talking to them about it. The mind-killer meme is fundamentally an encouragement to be mindful of that, especially of the nasty ideological feedback loops that it tends to imply, and a suggestion to focus on object-level issues where the feedback isn’t quite so intense.
One consequence of this is that political shifts happen at or above human timescales, as their subjects become things that established tribes notice they can fight over. If you happen to be a singularitarian, then, you probably believe that the kinds of technological and social changes that LW talks about will at some point—probably soon, possibly already—be moving faster than politics can keep up with. Speaking for myself, I expect anything that conventional legislatures or political parties say about AI to matter about as much as the RIAA did when they went after Napster, and still less once we’re in a position to be talking seriously about strong, friendly artificial intelligence.
More importantly from our perspective, though, anything conventional politics doesn’t care about yet is also something that we have a considerably better chance of talking about sanely. We may be—in fact, we’re certainly—in the territory of politics in the sense of subjects relevant to the future of the polis, but as long as identity considerations and politics-specific “conventional wisdom” stay relatively distant from our reasoning, we can expect our minds to remain relatively happy and unkilled.
Yeah, this comes up from time to time. My own approach to it is to (attempt as best as I can to) address the underlying policy question while avoiding language that gets associated with particular partisan groups.
For example, I might discuss how a Blue politician might oppose FAI because they value their social status, or how a Green politician might expect a Blue politician to oppose FAI for such a reason even though the Green politician is not driven purely by such motives, or whatever… rather than using other word-pairs like (Republican/Democrat), (liberal/conservative), (reactionary/progressive), or whatever.
If I get to a point in that conversation where the general points are clearly understood, and to make further progress I need to actually get into specifics about specific politicians and political parties, well, OK, I decide what to do when that happens. But that’s not where I start.
And I agree that the CEV version of that conversation is more difficult, and needs to be approached with more care to avoid being derailed by largely irrelevant partisan considerations, and that the same is true more generally about specific questions related to value tradeoffs and, even more generally, questions about where human values conflict with one another in the first place.
I don’t think the usual mantra of politics is the mind-killer is meant to avoid all political issues, although that would be one interpretation. Rather, there are two distinct observations: one is purely descriptive, that politics can be a mind-killer. The second is proscriptive- to possibly refrain when possible from discussing politics until our general level of rationality improves. Unfortunately, that’s fairly difficult, because many of these issues matter. Moreover, it connects with certain problems where counter-intuitive or contrarian ideas are seen as somehow less political than more mainstream ones.
That’s… an excellent way of putting it. Non-mainstream political “tribes” are considered “less political” precisely because they don’t stand any chance of actually winning elections in the real world, so they get a Meta-Contrarian Boost on the internet. The usual ones I see are anarchists, libertarians, and neo-reactionaries.
Empirically I don’t think this is true. Minority political tribes sometimes get a pass for organizing themselves around things that aren’t partisan issues, or are only minor partisan issues, in the mainstream—the Greens sometimes benefit from this in US discourse, although they’re a complicated and very regionally dependent case—but as soon as you stake out a position on a mainstream claim, even if your reasoning is very different from the norm, you should expect to be attacked as viciously as any mainstream wonk. I expect neoreaction, for example, would have met with a much less heated reception if it weren’t for its views on race.
Minority views do get a boost on the Internet, but I think that has more to do with the echo-chamber effects that it encourages. It’s far easier to find or collect a group of people that all agree with you on Reddit or Tumblr than it is out there in the slow, short-range world of blood and bone.
Considered where, and by whom? Because that is completely unlike my experience. On the Usenet groups rec.arts.sf.*, it was (I have not read Usenet for many years) absolutely standard that Progressive ideas were seen as non-political, while the merest hint of disagreement would immediately be piled on as “introducing politics to the discussion”. And the reactosphere is intensely aware that what they are talking is politics.
I generally agree with this post, but since people’s beliefs are evidence for how they change their beliefs in response to evidence, I would call it bias-inducing and usually tribal cheering instead of totally backwards.
If not “totally backwards” surely “orthogonal”. Why not a test that supplies it’s own evidence and asks the one being tested to come to a conclusion? Like the Amanda Knox case was for people here who hadn’t heard of it before reading about it here.
There are several situations where that’s not possible. Also it takes effort to test someone like that.
I wouldn’t call it orthogonal either. Rationality is about having correct beliefs, and I would label a belief-based litmus test rational to the extent it’s correct.
Writing a post about how $political_belief is a litmus test is probably a bad idea because of the reasons you mentioned.
Rationality is about have correct beliefs. But a single belief that has only two possible answers is never going to stand in for the entirety of a person’s belief structure. That’s why you have to look at the process by which a person forms beliefs to have any idea if they are rational.
Exactly. If there is any hope in using a list of beliefs as a test of rationality, it will need multiple items.
You know, IQ tests also don’t have a single question. Neither do any other personality tests.
OTOH the Cognitive Reflection Test has a shockingly low three questions and I’ve been told it’s surprisingly accurate.
I’d call it the “Paying-Good-Attention-While-Doing-Simple-Math Test”. :D
But yeah… I can imagine that something similarly simple could be an important part of rationality. Some simple task that predicts the ability to do more complex tasks of a similar type.
However, in that case the test will resemble a kind of puzzle, instead of pattern-matching “Do you agree with Greens?”
Specifically for updating, I can imagine a test where the person is gradually given more and more information; the initial information is an evidence of an outcome “A”, but most of the latter information is an evidence of an outcome “B”. The person is informally asked to make a guess soon after the beginning (when the reasonable answer is “A”), and at the end they are asked to provide a final answer. Some people would probably get stuck as “A”, and some would update to “B”. But the test would involve some small numbers, shapes, coins, etc.; not real-life examples.
I’ve seen experiments that tested this; I thought they were mentioned in Thinking and Deciding or Thinking Fast and Slow, but I didn’t see it in a quick check of either of those. If I recall the experimental setup correctly (I doubt I got the numbers right), they began with a sequence that was 80% red and 20% blue, which switched to being 80% blue and 20% red after n draws. The subjects’ estimate that the next draw would be red stayed above 50% for significantly longer than n draws from the second distribution, and some took until 2n or 3n draws from the second distribution to assign 50% chance to each, at which point almost two thirds of the examples they had seen were blue!
I dunno… people who do fine at the Wason selection task with ages and drinks get it wrong with numbers and colours. (I’m not sure whether that’s a bug or a feature.)
That seems to me like a reason not to test the skill on real-life examples.
We wouldn’t want a rationality test that a person can pass with original wording, but will fail if we replace “Republicans” by “Democrats”… or by Green aliens. We wouldn’t want the person to merely recognize logical fallacies when spoken by Republicans. This is in my opinion a risk with real-life examples. Is the example with drinking age easier because it is easier to imagine, or because it is something we already agree with?
Okay, I am curious here… what exactly would happen if we replaced the Wason selection task with something that uses words from real life (is less abstract), but is not an actual rule (therefore it cannot be answered using only previous experience)? For example: “Only dogs are allowed at jumping competitions, cats are not allowed. We have a) a dog going to unknown competition; b) a cat going to unknown competition; c) an unknown animal going to swimming competition, and d) an unknown animal going to jumping competition—which of these cases do you have to check thoroughly to make sure the rule is not broken?”
If I would want to estimate people rationality from beliefs I would look at whether the belief is nuanced. There are a lot of people who say irrational stuff such that they evidence we have for global warming is comparable to the evidence we have for evolution. In reality the p value doesn’t even approach the 5 sigma level that you need to validate a result about a new result in particle physics.
It’s just as irrational as being a global warming denier who thinks that p(global warming)<0.5.
Yet we do see smart people making both mistakes. You have smart people who claim that the evidence for global warming is comparable to evolution and you have smart people who are global warming deniers.
People don’t get mind killed by political issues because they are dumb. It might be completely rational for them because signaling is more important for them. If you want a useful metric do judge someone rationality don’t take something where group identities matter a good deal.
The metric is just too noisy because the person might get something from signaling group identity. I think the only reason to choose such a metric is because you get yourself mindkilled and want to label people who don’t belong to your tribe as irrational and seek some rationalisation for it.
As far as empirics go, collegue educated Republicans just have a higher rate of climate change denial than Republicans who didn’t go to collegue.
While we can discuss whether collegue causes people to be more rational it certainly correlates with it.
If you want to use beliefs to judge people rationality, calibrate the test. Give people ratioanlity quizes and quiz them for their beliefs. If you get strong correlations you have something that you can use. Don’t intellectually analyse the content of the beliefs and think about what rational people should believe if you want an effective metric.
RETRACTED: It wasn’t my intention to start another global warming debate.
Lots of insane beliefs are nuanced.
Requiring the same strength of evidence from climate science as from particle physics would be insane.
From Stuart’s post: “Of course, reverse stupidity isn’t intelligence: simply because one accepts AGW, doesn’t make one more rational.”
Choosing to signal wouldn’t be mindkill as it’s understood hjhink the only reason to choose such a metric is because you get yourself mindkilled and want to label people who don’t belong to your tribe as irrational and seek some rationalisation for it.
Labeling people seems to be exactly what you’re doing yourself here. I can think of at least three more reasons.
I think Stuart simply underestimated the local mindkill caused by global warming debate in other people, or failed to understand that local mindkill isn’t necessarily a good metric for irrationality. Neither of those require him to be mindkilled about the topic himself. One possibility is he failed to evaluate evidence of global warming himself and overestimated the probability of the relevant propositions.
You seem to be conflating intelligence and rationality in this comment. You probably know they’re not the same thing.
All this being said, I don’t agree with what Stuart was saying in his post. I have no opinion of global warming and haven’t read about it much.
What do you mean with “require”? If I say that climate science has the same strength of evidence as evolution than we can debate whether climate change does fulfill the 5 sigma criteria criteria.
I think it does, therefore the strength of evidence for climate change is not the same as the strength of evidence for evolution.
Why does it matter? It a X-risk that global warming doesn’t really exist and we do geoengineering that seriously wrecks our planet. That risk might be something like p=0.001 but it does exist. It’s greater than the risk of an asteroid destroying our civilisation in the next 100 years.
To the extend that one cares about X-risks it’s important to distinguish claims with 2-3 sigma from those who pass 5 sigmas. It’s just not the same level of evidence.
If we want to stay alive over the next hundred years it’s important that decision makers in our society don’t manuver us into an X-risk because they treat 2-3 sigma the same way as they treat treat 5 sigmas.
I don’t use the word intelligence in the comment you quote. I use it in another post as a proxy variable. I equate rationality for the ability to update your beliefs in order to win.
You used the words smart and dumb, I suppose that counts. I failed to understand most of your reply.
I mean you don’t need to be even nearly that certain for the findings to be actionable.
What’s the expected utility of that compared to the expected utility of AGW? If you’re too uncertain, why not just try to drastically reduce emissions instead of do major geoengineering? What’s the expected utility of reducing emissions?
If I ask “What’s the evidence for global warming being real?” in searching for an accurate description of the world. Having accurate maps of the world is useful.
In the above example, saying that the evidence for global warming is like that for evolution is like claiming the moon is made of cheese.
The belief might help you to convince people to reduce emissions. Believing that the moon is made of cheese might help you to discourage people from going to the moon.
If the reason that someone advocates the ridiculous claim that the evidence for global warming is comparable to that for evolution, is that it helps him convince people to lower emission that person is mindkilled by his politics.
Right, because our political leaders excel at doing rational good expected utility comparisions… Memes exist in the real world. They have effects. Promoting false beliefs about the certainity of science has dangers.
I’m not in the position to have the power to choose that the world drastically reduces emissions or whether it does major geoengineering and scientists aren’t either. Scientists do have a social responsiblity to promote accurate beliefs about the world.
Whether or not we should reduce emissions is a different question. If you can’t mentally separate: “Should we reduce emissions” from “What’s the evidence for global warming?” you likely mindkilled about the second question and hold beliefs that aren’t accurate descriptions of reality.
The current understanding of climate sensitivity is that since Carbon Dioxide gas will remain in the upper atmosphere for decades (and possibly centuries) even a complete halt on emissions will not avert warming predicted for the next century or so. And the models currently favored have pretty dire predictions for that level of warming, even if they’re less severe than the alternative.
The only realistic solution, and naturally the one most strongly opposed by environmental groups, is solar radiation management. This would be very expensive, about $700M a year according to David Keith, and has potential risks which should be tested before any implementation plan. So not a silver bullet, but still much cheaper and safer in the long run than the standard environmental agenda even according to their own data.
(Note: I am assuming for the sake of argument that current climate models are accurate, but that is an assumption which should be questioned. Climate modeling is still in it’s infancy and most existing models have difficulty with predictions even as close as a decade out. Warming is probably happening but that does not mean that any given prediction of warming is accurate, for reasons which should be obvious.)
Methane has a shorter lifetime, though (though my five minutes’ research tells me we’ve already stopped increasing methane emissions).
Are you saying that solar radiation management is an alternative to long-term emissions reduction? Or that, in addition to eventually tapering off greenhouse gas emissions, we’re going to have to do something to keep temperatures down, and the best option is solar radiation management?
(edit: apparently I wrote social radiation management)
Reducing emissions is a good goal, but energy needs will continue to increase even as we decrease the number of tons of carbon dioxide per kWh. As the population increases and becomes more wealthy there’s not much we can do but put out more carbon dioxide; that’s one of the reasons people bent on lowering world population and wealth have attached themselves to the environmental movement.
If the stigma against nuclear power goes away, or the technological issues which make speculative energy sources like wind/solar/fusion unprofitable are resolved, we could see a bigger dip but even then the century-long trend will probably be one of increase. SRM is the most realistic way I can think of to head off serious disasters until then.
The whole point is how you change your beliefs in response to new evidence.
Of course the general concept of using a belief as a litmus test for rationality is foolish. But frankly, it’s not possible at this point to have not been introduced to evidence about human-caused global warming. The people to which this test would be applied have been introduced to this new evidence and already failed to update.
And if someone lives in such a secluded bubble of information that they are truly getting information that would lead a rationalist to decry AGW, I think it safe to say that that person is probably not a rationalist. Someone in such a bubble would have no impetus to become a rationalist in the first place.