“A random walk, in retrospect, looks like like directional movement at a speed of .
The average distance from the starting point is close to after n random steps (in 1 dimension). But I’d characterize that as a speed of . Or you could say ”… looks like a directional movement of distance ”.
I expect that’s about not trusting the foreigners who did the clinical trials (I have heard this)[1], and not so much about expecting that Americans are biologically different from foreigners.
Specifically, someone with some knowledge told me that the FDA knows that there are some countries where the trials are completely untrustworthy. And that there was a political decision where they said, “If we disallow trials from some countries but not others, there will be much complaining and we’ll probably be called racists”, and solved the issue by disallowing trials from all foreign countries.
Let’s see what the base rate for murder is. After some googling… Since the “clearance rate” for homicides is 50% (as of a recent year), even if we know there were N murders, it’s hard to say how many distinct murderers there were. But some source says it’s a small percentage of murderers who kill again, so let’s just assume that N murders = N murderers. Both “taking the homicide rate (7.5 per 100k per year in 2022) and multiplying by a lifetime (we’ll say 80 years)”, and “googling for the percentage of deaths that are caused by homicide”, yield roughly 0.6%. So 6 per thousand.
I haven’t kept count of the numerator, but if you were going to argue that the murderer rate is no different than among the American general population, the denominator would need to be in the high hundreds at least. (It might also be interesting to control for other demographics of rationalists.)
A better counterargument would be that Ziz’s associates just happened to be trans, because that’s a shared quality that can be quite important to people in ways that may make them tend to stick together, which is also true of, say, having a shared hobby like Magic: The Gathering. So when Ziz became a murder cult leader, and several associates committed murder, then that shared quality ends up being highly correlated with murder, and this could happen just as easily with any other …
But that’s not quite true. Ziz’s writings have talked a lot about trans stuff, and I have the impression (it’s been a couple of years since I read the blog, and this might not necessarily be written too explicitly) that Ziz made a point of seeking out trans people or, probably most especially, pre-trans or gender-questioning people. (At least, that was the result; it could have happened for other reasons; but I think there was an optimization process pointing in this direction.) Because someone in that stage, in the process of majorly rethinking their identity and how they relate to the world, possibly experiencing significant gender dysphoria and associated issues… Is probably much more vulnerable than average to a cult leader saying, “I understand people like you, I can tell you all kinds of things (probably many true things!) about our kind, I have this whole philosophy that I’ve written out, and you should spend lots of time with me as I explain it to you… And by the way the philosophy says the world is mostly evil and we should lose any aversion to violence and also extreme violence is obligatory in certain cases.” It is much less plausible that one could do all that with Magic: The Gathering buddies.[1]
So, if this model is correct, that (a) Ziz specifically targeted early-stage trans people and (b) early-stage trans people are especially vulnerable to cult recruitment[2], then the results are less surprising. A compassionate response would be to do something like make a point of warning young trans and maybe-trans people about cult recruitment in general, and perhaps about Ziz specifically (though if most haven’t already heard about Ziz, it might be best to keep it that way). I guess a less-compassionate response is to exclude them completely.
A somewhat similar crop of potential recruits, in terms of psychological vulnerability, would be, well, psychiatric patients. Have there been therapist cults? Oh yes, Dianetics and things derived from it such as Re-evaluation Counseling come to mind. (Was L. Ron Hubbard himself a therapist? He did apparently volunteer at a mental health clinic, but I’m not sure how much professional training he got.)
Research question: Are there other cases of cults that targeted trans / gender-questioning people? I think I’ve heard things, but don’t have good pointers to find out frequency and severity.
I’d point out that the magnitude of the “exploitation” is the magnitude of the incentive for market players to find the better solution. If Bob is the one guy for whom making him wear a tuxedo isn’t worthwhile, and if it’s close to worthwhile—e.g. him wearing it produces $4000 of value to the company over the time he wears it—then that’s $1000 being left on the table. If there are 100 employees being “exploited” like Bob, for whom making them wear tuxedos is extremely wasteful—say it produces only $100 of value for them to wear tuxedos—then there is $490,000 being left on the table, which may be enough to justify making a new company if the existing one is for some reason stubborn about the issue. (Also, the more employees there are in that situation, the more likely it is that someone will complain and someone else will notice the opportunity.)
Some related scenarios are discussed in my post here, e.g. when popularity ≈ beauty + substance, and if popularity and beauty are readily apparent then you can estimate substance.
There had been a study comparing the effects of moderate protests vs extreme protests, in hypothetical situations (the study participants would e.g. read a fictional article describing the activities of a protest group), and concluded that “Extreme Protest Actions Reduce Popular Support for Social Movements”:
How do protest actions impact public support for social movements? Here we test the claim that extreme protest actions—protest behaviors perceived to be harmful to others, highly disruptive, or both—typically reduce support for social movements. Across 6 experiments, including 3 that were preregistered, participants indicated less support for social movements that used more extreme protest actions. This result obtained across a variety of movements (e.g., animal rights, anti-Trump, anti-abortion) and extreme protest actions (e.g., blocking highways, vandalizing property). Further, in 5 of 6 studies, negative reactions to extreme protest actions also led participants to support the movement’s central cause less, and these effects were largely independent of individuals’ prior ideology or views on the issue. In all studies we found effects were driven by diminished social identification with the movement. In Studies 4-6, serial mediation analyses detailed a more in-depth model: observers viewed extreme protest actions to be immoral, reducing observers’ emotional connection to the movement and, in turn, reducing identification with and support for the movement. Taken together with prior research showing that extreme protest actions can be effective for applying pressure to institutions and raising awareness of movements, these findings suggest an activist’s dilemma, in which the same protest actions that may offer certain benefits are also likely to undermine popular support for social movements.
It could be that moderate protests are useful. It would be an interesting test of a group: whether it’s able to consistently avoid extreme behavior.
Smart people are often too arrogant and proud, and know too much.
I thought that might be the case. If you looked at GPT-3 or 3.5, then, the higher the quality of your own work, the less helpful (and, potentially, the more destructive and disruptive) it is to substitute in the LLM’s work; so higher IQ in these early years of LLMs may correlate with dismissing them and having little experience using them.
But this is a temporary effect. Those who initially dismissed LLMs will eventually come round; and, among younger people, especially as LLMs get better, higher-IQ people who try LLMs for the first time will find them worthwhile and use them just as much as their peers. And if you have two people who have both spent N hours using the same LLM for the same purposes, higher IQ will help, all else being equal.
Of course, if you’re simply reporting a correlation you observe, then all else is likely not equal. Please think about selection effects, such as those described here.
Using LLMs is an intellectual skill. I would be astonished if IQ was not pretty helpful for that.
For editing adults, it is a good point that lots of them might find a personality tweak very useful, and e.g. if it gave them a big bump in motivation, that would likely be worth more than, say, 5-10 IQ points. An adult is in a good position to tell what’s the delta between their current personality and what might be ideal for their situation.
Deliberately tweaking personality does raise some “dual use” issues. Is there a set of genes that makes someone very unlikely to leave their abusive cult, or makes them loyal obedient citizens to their tyrannical government, or makes them never join the hated outgroup political party? I would be pretty on board with a norm of not doing research into that. Basic “Are there genes that cause personality disorders that ~everyone agrees are bad?” is fine; “motivation” as one undifferentiated category seems fine; Big 5 traits … have some known correlations with political alignment, which brings it into territory I’m not very comfortable with, but if it goes no farther that it might be fine.
On a quick skim, an element that seems to be missing is that having emotions which cause you to behave ‘irrationally’ can in fact be beneficial from a rational perspective.
For example, if everyone knows that, when someone does you a favor, you’ll feel obligated to find some way to repay them, and when someone injures you, you’ll feel driven to inflict vengeance upon them even at great cost to yourself—if everyone knows this about you, then they’ll be more likely to do you favors and less likely to injure you, and your expected payoffs are probably higher than if you were 100% “rational” and everyone knew it. I believe this is in fact why we have the emotions of gratitude and anger, and I think various animals have something resembling them. Put it this way: carrying out threats and promises is “irrational” by definition, but making your brain into a thing that will carry out threats and promises may be very rational.
So you could call these emotions “irrational” or the thoughts they lead to “biased”, but I think that (a) likely pushes your thinking in the wrong direction in general, and (b) gives you no guidance on what “irrational” emotions are likely to exist.
What is categorized as “peer pressure” here? Explicit threats to report you to authorities if you don’t conform? I’m guessing not. But how about implicit threats? What if you’ve heard (or read in the news) stories about people who don’t conform—in ways moderately but not hugely more extreme than you—having their careers ruined? In any situation that you could call “peer pressure”, I imagine there’s always at least the possibility of some level of social exclusion.
The defining questions for that aspect would appear to be “Do you believe that you would face serious risk of punishment for not conforming?” and “Would a reasonable person in your situation believe the same?”. Which don’t necessarily have the same answer. It might, indeed, be that people whom you observe to be “conformist” are the ones who are oversensitive to the risk of social exclusion.
The thing that comes to mind, when I think of “formidable master of rationality”, is a highly experienced engineer trying to debug problems, especially high-urgency problems that the normal customer support teams haven’t been able to handle. You have a fresh phenomenon, which the creators of the existing product apparently didn’t anticipate (or if they did, they didn’t think it worth adding functionality to handle it), which casts doubt on existing diagnostic systems. You have priors on which tools are likely to still work, priors on which underlying problems are likely to cause which symptoms; tests you can try, each of which has its own cost and range of likely outcomes, and some of which you might invent on the spot; all of these lead to updating your probability distribution over what the underlying problem might be.
Medical diagnostics, as illustrated by Dr. House, can be similar, although I suspect the frequency of “inventing new tests to diagnose a never-before-seen problem” is lower there.
One argument I’ve encountered is that sentient creatures are precisely those creatures that we can form cooperative agreements with. (Counter-argument: one might think that e.g. the relationship with a pet is also a cooperative one [perhaps more obviously if you train them to do something important, and you feed them], while also thinking that pets aren’t sentient.)
Another is that some people’s approach to the Prisoner’s Dilemma is to decide “Anyone who’s sufficiently similar to me can be expected to make the same choice as me, and it’s best for all of us if we cooperate, so I’ll cooperate when encountering them”; and some of them may figure that sentience alone is sufficient similarity.
So, the arithmetic and geometric mean agree when the inputs are equal, and, the more unequal they are, the lower the geometric mean is.
I note that the subtests have ceilings, which puts a limit on how much any one can skew the result. Like, if you have 10 subtests, and the max score is something like 150, then presumably each test has a max score of 15 points. If we imagine someone gets five 7s and five 13s (a moderately unbalanced set of abilities), then the geometric mean is 9.54, while the arithmetic mean is 10. So, even if someone were confused about whether the IQ test was using a geometric or an arithmetic mean, does it make a large difference in practice?
The people you’re arguing against, is it actually a crux for them? Do they think IQ tests are totally invalid because they’re using an arithmetic mean, but actually they should realize it’s more like a geometric mean and then they’d agree IQ tests are great?
1. IQ scores do not measure even close to all cognitive abilities and realistically could never do that.
Well, the original statement was “sums together cognitive abilities” and didn’t use the word “all”, and I, at least, saw no reason to assume it. If you’re going to say something along the lines of “Well, I’ve tried to have reasonable discussions with these people, but they have these insane views”, that seems like a good time to be careful about how you represent those views.
2. Many of the abilities that IQ scores weight highly are practically unimportant.
Are you talking about direct measurement, or what they correlate with? Because, certainly, things like anagramming a word have almost no practical application, but I think it’s intended to (and does) correlate with language ability. But in any case, the truth value of the statement that IQ is “an index that sums together cognitive abilities” is unaffected by whether those abilities are useful ones.
Perhaps you have some idea of a holistic view, of which that statement is only a part, and maybe that holistic view contains other statements which are in fact insane, and you’re attacking that view, but… in the spirit of this post, I would recommend confining your attacks to specific statements rather than to other claims that you think correlate with those statements.
3. Differential-psychology tests are in practice more like log scales than like linear scales, so “sums” are more like products than like actual suns; even if you are absurdly good at one thing, you’re going to have a hard time competing with someone in IQ if they are moderately better at many things.
I wonder how large a difference this makes in practice. So if we run with your claim here, it seems like your conclusion would be… that IQ tests combine the subtest scores in the wrong way, and are less accurate than they should be for people with very uneven abilities? Is that your position? At any rate, even if the numbers are logarithms, it’s still correct to say that the test is adding them up, and I don’t consider that good grounds for calling it “insane” for people to consider it addition.
thinks of IQ as an index that sums together cognitive abilities
Is this part not technically true? IQ tests tend to have a bunch of subtests intended to measure different cognitive abilities, and you add up—or average, which is adding up and dividing by a constant—your scores on each subtest. For example (bold added):
The current version of the test, the WAIS-IV, which was released in 2008, is composed of 10 core subtests and five supplemental subtests, with the 10 core subtests yielding scaled scores that sum to derive the Full Scale IQ.
Interesting. The natural approach is to imagine that you just have a 3-sided die with 2, 4, 6 on the sides, and if you do that, then I compute A = 12 and B = 6[1]. But, as the top Reddit comment’s edit points out, the difference between that problem and the one you posed is that your version heavily weights the probability towards short sequences—that weighting being 1/2^n for a sequence of length n. (Note that the numbers I got, A=12 and B=6, are so much higher than the A≈2.7 and B=3 you get.) It’s an interesting selection effect.
The thing is that, if you roll a 6 and then a non-6, in an “A” sequence you’re likely to just die due to rolling an odd number before you succeed in getting the double 6, and thus exclude the sequence from the surviving set; whereas in a “B” sequence there’s a much higher chance you’ll roll a 6 before dying, and thus include this longer “sequence of 3+ rolls” in the set.
To illustrate with an extreme version, consider:
A: The expected number of rolls of a fair die until you roll two 6s in a row, given that you succeed in doing this. You ragequit if it takes more than two rolls.
Obviously that’s one way to reduce A to 2.
Excluding odd rolls completely, so the die has a 1⁄3 chance of rolling 6 and a 2⁄3 chance of rolling an even number that’s not 6, we have:
A = 1 + 1⁄3 * A2 + 2⁄3 * A
Where A2 represents “the expected number of die rolls until you get two 6′s in a row, given that the last roll was a 6”. Subtraction and multiplication then yields:
A = 3 + A2
And if we consider rolling a die from the A2 state, we get:
A2 = 1 + 1⁄3 * 0 + 2⁄3 * A
= 1 + 2⁄3 * A
Substituting:
A = 3 + 1 + 2⁄3 * A
=> (subtract)
1⁄3 * A = 4
=> (multiply)
A = 12
For B, a similar approach yields the equations:
B = 1 + 1⁄3 * B2 + 2⁄3 * B
B2 = 1 + 1⁄3 * 0 + 2⁄3 * B2
And the reader may solve for B = 6.
I can also come up with a story where obviously it’s cheaper and more effective to disable all of the nuclear weapons than it is to take over the world, so why would the AI do the second thing?
Erm… For preventing nuclear war on the scale of decades… I don’t know what you have in mind for how it would disable all the nukes, but a one-off breaking of all the firing mechanisms isn’t going to work. They could just repair/replace that once they discovered the problem. You could imagine some more drastic thing like blowing up the conventional explosives on the missiles so as to utterly ruin them, but in a way that doesn’t trigger the big chain reaction. But my impression is that, if you have a pile of weapons-grade uranium, then it’s reasonably simple to make a bomb out of it, and since uranium is an element, no conventional explosion can eliminate that from the debris. Maybe you can melt it, mix it with other stuff, and make it super-impure?
But even then, the U.S. and Russia probably have stockpiles of weapons-grade uranium. I suspect they could make nukes out of that within a few months. You would have to ruin all the stockpiles too.
And then there’s the possibility of mining more uranium and enriching it; I feel like this would take a few years at most, possibly much less if one threw a bunch of resources into rushing it. Would you ruin all uranium mines in the world somehow?
No, it seems to me that the only ways to reliably rule out nuclear war involve either using overwhelming physical force to prevent people from using or making nukes (like a drone army watching all the uranium stockpiles), or being able to reliably persuade the governments of all nuclear powers in the world to disarm and never make any new nukes. The power to do either of these things seems tantamount to the power to take over the world.
why most perfect algorithms that recreate a strawberry on the molecular level destroy the planet as well.
Phrased like this, the answer that comes to mind is “Well, this requires at least a few decades’ worth of advances in materials science and nanotechnology and such, plus a lot of expensive equipment that doesn’t exist today, and e.g. if you want this to happen with high probability, you need to be sure that civilization isn’t wrecked by nuclear war or other threats in upcoming decades, so if you come up with a way of taking over the world that has higher certainty than leaving humanity to its own devices, then that becomes the best plan.” Classic instrumental convergence, in other words.
The political version of the question isn’t functionally the same as the skin cream version, because the former isn’t a randomized intervention—cities that decided to add gun control laws seem likely to have other crime-related events and law changes at the same time, which could produce a spurious result in either direction. So it’s quite reasonable to say “My opinion is determined by my priors and the evidence didn’t appreciably affect my position.”
@Friendly Monkey , I’m replying to your reaction:
There are people who require multiple methods of persuasion before they act in the way you want. One category is decisionmakers for an organization, who have actually been persuaded by intimidation, but they can’t just say that, because they would look weak and possibly as though they’re defecting against the organization or its aims, so they need to sell it as some high-minded decision they’ve come to of their own accord. Or it could be the reverse: decisionmakers who are persuaded by your ideological arguments, but are funded / otherwise kept in charge by those who don’t care or have contempt for the ideology, so they need to sell it to their funders (presumably in private) as, “Hey, look, let’s be realistic here, if we do this then they’ll do that and we absolutely can’t afford that. But if we do this other thing, that won’t happen, and would it really be so bad? And we’ll tell the public that recent events have made us realize how important [...]”.
In both cases, it’s essential for you to have someone doing the intimidation and someone publicizing the high-minded arguments, and usually it works best if these are different people. (For example, if an intellectual who is respected by the mainstream (but only agreed with by a minority) starts making threats, that seems likely to lose them mainstream acceptance—such an ugly thing to be involved with, carrying out the threats even more so—and thus, for that reason among others, making the threats credible is more difficult than it would be for a thug who has nothing to lose.)
So, for those decisionmakers, if you have intimidation but not a public-friendly face (a book-publishing intellectual, an organization doing charity events, etc.), you get nothing, and if you have the friendly face but not intimidation, you also get nothing, but if you have both, then you win their support. It’s not a matter of “intimidation and friendliness each independently get diminishing marginal returns, and if you over-invest in one or even saturate it then that’s inefficient”; rather, you need both to have any success.
Incidentally, although violence was the subject above, I’ve used “intimidation”, which may be interpreted to also cover things like social shaming or threatening to arbitrarily cancel business deals. That makes the above patterns cover a lot of things done in recent years.
Another aspect: Frequently, the group doing the intimidation will claim it’s justified. One man’s threat of aggressive violence may be another man’s statement that he’ll act in self-defense or justified punishment, if they have different theories of rights, or perhaps disagreement about what happened.
The more mainstream-friendly group who is on their side… If they want to defend the behavior, then, depending on the circumstances, they have lots of options for their official position: the ideological, “It’s justified” or “It’s an overreaction but you do have to understand where they’re coming from”; the conversation-tactical, “Hey, look over there! Something more important is happening” or “Anyone who complains about this has contemptible traits XYZ and should be attacked”; the associational, “It wasn’t our people”, or even “It was a false flag”; the evidentiary, “It’s not as bad as they claim”, occasionally even “It didn’t happen”; and so on. (Any of these stances might have been chosen honestly, and might be correct, but, especially for the most sophisticated and the most ideological, one’s priors should be skeptical. Sometimes “Bounded Distrust” is applicable; it can be interesting to think through “If they could have taken and defended stance A, they probably would have, so the fact that they picked stance B tells me …”.)