localdeity

Karma: 2,333

localdeity 16 Dec 2024 21:40 UTC
2 points
0
in reply to: gwern’s comment on: Significantly Enhancing Adult Intelligence With Gene Editing May Be Possible
Smart people are often too arrogant and proud, and know too much.
I thought that might be the case. If you looked at GPT-3 or 3.5, then, the higher the quality of your own work, the less helpful (and, potentially, the more destructive and disruptive) it is to substitute in the LLM’s work; so higher IQ in these early years of LLMs may correlate with dismissing them and having little experience using them.
But this is a temporary effect. Those who initially dismissed LLMs will eventually come round; and, among younger people, especially as LLMs get better, higher-IQ people who try LLMs for the first time will find them worthwhile and use them just as much as their peers. And if you have two people who have both spent N hours using the same LLM for the same purposes, higher IQ will help, all else being equal.
Of course, if you’re simply reporting a correlation you observe, then all else is likely not equal. Please think about selection effects, such as those described here.

localdeity 16 Dec 2024 7:48 UTC
5 points
3
in reply to: gwern’s comment on: Significantly Enhancing Adult Intelligence With Gene Editing May Be Possible
Using LLMs is an intellectual skill. I would be astonished if IQ was not pretty helpful for that.
For editing adults, it is a good point that lots of them might find a personality tweak very useful, and e.g. if it gave them a big bump in motivation, that would likely be worth more than, say, 5-10 IQ points. An adult is in a good position to tell what’s the delta between their current personality and what might be ideal for their situation.
Deliberately tweaking personality does raise some “dual use” issues. Is there a set of genes that makes someone very unlikely to leave their abusive cult, or makes them loyal obedient citizens to their tyrannical government, or makes them never join the hated outgroup political party? I would be pretty on board with a norm of not doing research into that. Basic “Are there genes that cause personality disorders that ~everyone agrees are bad?” is fine; “motivation” as one undifferentiated category seems fine; Big 5 traits … have some known correlations with political alignment, which brings it into territory I’m not very comfortable with, but if it goes no farther that it might be fine.

localdeity 13 Dec 2024 1:44 UTC
3 points
0
on: Representing Irrationality in Game Theory
On a quick skim, an element that seems to be missing is that having emotions which cause you to behave ‘irrationally’ can in fact be beneficial from a rational perspective.
For example, if everyone knows that, when someone does you a favor, you’ll feel obligated to find some way to repay them, and when someone injures you, you’ll feel driven to inflict vengeance upon them even at great cost to yourself—if everyone knows this about you, then they’ll be more likely to do you favors and less likely to injure you, and your expected payoffs are probably higher than if you were 100% “rational” and everyone knew it. I believe this is in fact why we have the emotions of gratitude and anger, and I think various animals have something resembling them. Put it this way: carrying out threats and promises is “irrational” by definition, but making your brain into a thing that will carry out threats and promises may be very rational.
So you could call these emotions “irrational” or the thoughts they lead to “biased”, but I think that (a) likely pushes your thinking in the wrong direction in general, and (b) gives you no guidance on what “irrational” emotions are likely to exist.

localdeity 9 Dec 2024 22:31 UTC
7 points
2
in reply to: sarahconstantin’s comment on: sarahconstantin’s Shortform
What is categorized as “peer pressure” here? Explicit threats to report you to authorities if you don’t conform? I’m guessing not. But how about implicit threats? What if you’ve heard (or read in the news) stories about people who don’t conform—in ways moderately but not hugely more extreme than you—having their careers ruined? In any situation that you could call “peer pressure”, I imagine there’s always at least the possibility of some level of social exclusion.
The defining questions for that aspect would appear to be “Do you believe that you would face serious risk of punishment for not conforming?” and “Would a reasonable person in your situation believe the same?”. Which don’t necessarily have the same answer. It might, indeed, be that people whom you observe to be “conformist” are the ones who are oversensitive to the risk of social exclusion.

localdeity 3 Dec 2024 20:16 UTC
2 points
0
on: A Sense That More Is Possible
The thing that comes to mind, when I think of “formidable master of rationality”, is a highly experienced engineer trying to debug problems, especially high-urgency problems that the normal customer support teams haven’t been able to handle. You have a fresh phenomenon, which the creators of the existing product apparently didn’t anticipate (or if they did, they didn’t think it worth adding functionality to handle it), which casts doubt on existing diagnostic systems. You have priors on which tools are likely to still work, priors on which underlying problems are likely to cause which symptoms; tests you can try, each of which has its own cost and range of likely outcomes, and some of which you might invent on the spot; all of these lead to updating your probability distribution over what the underlying problem might be.
Medical diagnostics, as illustrated by Dr. House, can be similar, although I suspect the frequency of “inventing new tests to diagnose a never-before-seen problem” is lower there.

localdeity 26 Nov 2024 10:06 UTC
3 points
0
in reply to: Sinclair Chen’s comment on: Sinclair Chen’s Shortform
One argument I’ve encountered is that sentient creatures are precisely those creatures that we can form cooperative agreements with. (Counter-argument: one might think that e.g. the relationship with a pet is also a cooperative one [perhaps more obviously if you train them to do something important, and you feed them], while also thinking that pets aren’t sentient.)
Another is that some people’s approach to the Prisoner’s Dilemma is to decide “Anyone who’s sufficiently similar to me can be expected to make the same choice as me, and it’s best for all of us if we cooperate, so I’ll cooperate when encountering them”; and some of them may figure that sentience alone is sufficient similarity.

localdeity 25 Nov 2024 16:59 UTC
2 points
0
in reply to: tailcalled’s comment on: Crosspost: Developing the middle ground on polarized topics
So, the arithmetic and geometric mean agree when the inputs are equal, and, the more unequal they are, the lower the geometric mean is.
I note that the subtests have ceilings, which puts a limit on how much any one can skew the result. Like, if you have 10 subtests, and the max score is something like 150, then presumably each test has a max score of 15 points. If we imagine someone gets five 7s and five 13s (a moderately unbalanced set of abilities), then the geometric mean is 9.54, while the arithmetic mean is 10. So, even if someone were confused about whether the IQ test was using a geometric or an arithmetic mean, does it make a large difference in practice?
The people you’re arguing against, is it actually a crux for them? Do they think IQ tests are totally invalid because they’re using an arithmetic mean, but actually they should realize it’s more like a geometric mean and then they’d agree IQ tests are great?

localdeity 25 Nov 2024 16:12 UTC
5 points
1
in reply to: tailcalled’s comment on: Crosspost: Developing the middle ground on polarized topics
1. IQ scores do not measure even close to all cognitive abilities and realistically could never do that.
Well, the original statement was “sums together cognitive abilities” and didn’t use the word “all”, and I, at least, saw no reason to assume it. If you’re going to say something along the lines of “Well, I’ve tried to have reasonable discussions with these people, but they have these insane views”, that seems like a good time to be careful about how you represent those views.
2. Many of the abilities that IQ scores weight highly are practically unimportant.
Are you talking about direct measurement, or what they correlate with? Because, certainly, things like anagramming a word have almost no practical application, but I think it’s intended to (and does) correlate with language ability. But in any case, the truth value of the statement that IQ is “an index that sums together cognitive abilities” is unaffected by whether those abilities are useful ones.
Perhaps you have some idea of a holistic view, of which that statement is only a part, and maybe that holistic view contains other statements which are in fact insane, and you’re attacking that view, but… in the spirit of this post, I would recommend confining your attacks to specific statements rather than to other claims that you think correlate with those statements.
3. Differential-psychology tests are in practice more like log scales than like linear scales, so “sums” are more like products than like actual suns; even if you are absurdly good at one thing, you’re going to have a hard time competing with someone in IQ if they are moderately better at many things.
I wonder how large a difference this makes in practice. So if we run with your claim here, it seems like your conclusion would be… that IQ tests combine the subtest scores in the wrong way, and are less accurate than they should be for people with very uneven abilities? Is that your position? At any rate, even if the numbers are logarithms, it’s still correct to say that the test is adding them up, and I don’t consider that good grounds for calling it “insane” for people to consider it addition.

localdeity 25 Nov 2024 15:07 UTC
3 points
0
in reply to: tailcalled’s comment on: Crosspost: Developing the middle ground on polarized topics
thinks of IQ as an index that sums together cognitive abilities
Is this part not technically true? IQ tests tend to have a bunch of subtests intended to measure different cognitive abilities, and you add up—or average, which is adding up and dividing by a constant—your scores on each subtest. For example (bold added):
The current version of the test, the WAIS-IV, which was released in 2008, is composed of 10 core subtests and five supplemental subtests, with the 10 core subtests yielding scaled scores that sum to derive the Full Scale IQ.

Perils of Generalizing from One’s Social Group

localdeity24 Nov 2024 15:31 UTC

64 points

1 comment3 min readLW link

localdeity 22 Nov 2024 15:22 UTC
53 points
14
on: A very strange probability paradox
Interesting. The natural approach is to imagine that you just have a 3-sided die with 2, 4, 6 on the sides, and if you do that, then I compute A = 12 and B = 6^[1]. But, as the top Reddit comment’s edit points out, the difference between that problem and the one you posed is that your version heavily weights the probability towards short sequences—that weighting being 1/2^n for a sequence of length n. (Note that the numbers I got, A=12 and B=6, are so much higher than the A≈2.7 and B=3 you get.) It’s an interesting selection effect.
The thing is that, if you roll a 6 and then a non-6, in an “A” sequence you’re likely to just die due to rolling an odd number before you succeed in getting the double 6, and thus exclude the sequence from the surviving set; whereas in a “B” sequence there’s a much higher chance you’ll roll a 6 before dying, and thus include this longer “sequence of 3+ rolls” in the set.
To illustrate with an extreme version, consider:
A: The expected number of rolls of a fair die until you roll two 6s in a row, given that you succeed in doing this. You ragequit if it takes more than two rolls.
Obviously that’s one way to reduce A to 2.
1. ^
  Excluding odd rolls completely, so the die has a ¹⁄₃ chance of rolling 6 and a ²⁄₃ chance of rolling an even number that’s not 6, we have:
  A = 1 + ¹⁄₃ * A2 + ²⁄₃ * A
  Where A2 represents “the expected number of die rolls until you get two 6′s in a row, given that the last roll was a 6”. Subtraction and multiplication then yields:
  A = 3 + A2
  And if we consider rolling a die from the A2 state, we get:
  A2 = 1 + ¹⁄₃ * 0 + ²⁄₃ * A
  = 1 + ²⁄₃ * A
  Substituting:
  A = 3 + 1 + ²⁄₃ * A
  => (subtract)
  ¹⁄₃ * A = 4
  => (multiply)
  A = 12
  For B, a similar approach yields the equations:
  B = 1 + ¹⁄₃ * B2 + ²⁄₃ * B
  B2 = 1 + ¹⁄₃ * 0 + ²⁄₃ * B2
  And the reader may solve for B = 6.

localdeity 14 Nov 2024 15:35 UTC
2 points
0
in reply to: Tahp’s comment on: Thoughts after the Wolfram and Yudkowsky discussion
I can also come up with a story where obviously it’s cheaper and more effective to disable all of the nuclear weapons than it is to take over the world, so why would the AI do the second thing?
Erm… For preventing nuclear war on the scale of decades… I don’t know what you have in mind for how it would disable all the nukes, but a one-off breaking of all the firing mechanisms isn’t going to work. They could just repair/replace that once they discovered the problem. You could imagine some more drastic thing like blowing up the conventional explosives on the missiles so as to utterly ruin them, but in a way that doesn’t trigger the big chain reaction. But my impression is that, if you have a pile of weapons-grade uranium, then it’s reasonably simple to make a bomb out of it, and since uranium is an element, no conventional explosion can eliminate that from the debris. Maybe you can melt it, mix it with other stuff, and make it super-impure?
But even then, the U.S. and Russia probably have stockpiles of weapons-grade uranium. I suspect they could make nukes out of that within a few months. You would have to ruin all the stockpiles too.
And then there’s the possibility of mining more uranium and enriching it; I feel like this would take a few years at most, possibly much less if one threw a bunch of resources into rushing it. Would you ruin all uranium mines in the world somehow?
No, it seems to me that the only ways to reliably rule out nuclear war involve either using overwhelming physical force to prevent people from using or making nukes (like a drone army watching all the uranium stockpiles), or being able to reliably persuade the governments of all nuclear powers in the world to disarm and never make any new nukes. The power to do either of these things seems tantamount to the power to take over the world.

localdeity 14 Nov 2024 4:04 UTC
3 points
1
on: Thoughts after the Wolfram and Yudkowsky discussion
why most perfect algorithms that recreate a strawberry on the molecular level destroy the planet as well.
Phrased like this, the answer that comes to mind is “Well, this requires at least a few decades’ worth of advances in materials science and nanotechnology and such, plus a lot of expensive equipment that doesn’t exist today, and e.g. if you want this to happen with high probability, you need to be sure that civilization isn’t wrecked by nuclear war or other threats in upcoming decades, so if you come up with a way of taking over the world that has higher certainty than leaving humanity to its own devices, then that becomes the best plan.” Classic instrumental convergence, in other words.

localdeity 5 Nov 2024 12:22 UTC
2 points
0
in reply to: Matt Goldenberg’s comment on: Matt Goldenberg’s Short Term Feed
The political version of the question isn’t functionally the same as the skin cream version, because the former isn’t a randomized intervention—cities that decided to add gun control laws seem likely to have other crime-related events and law changes at the same time, which could produce a spurious result in either direction. So it’s quite reasonable to say “My opinion is determined by my priors and the evidence didn’t appreciably affect my position.”

localdeity 27 Oct 2024 2:11 UTC
31 points
10
on: localdeity’s Shortform
90% awful idea: “Genetic diversity” in computer programs for resistance to large-scale cyberattacks.

The problem: Once someone has figured out the right security hole in Tesla’s software (and, say, broken into a server used to deliver software updates), they can use this to install their malicious code into all 5 million Teslas in the field (or maybe just one model, so perhaps 1 million cars), and probably make them all crash simultaneously and cause a catastrophe.
The solution: There will probably come a point where we can go through the codebase and pick random functions and say, “Claude, write a specification of what this function does”, and then “Claude, take this specification and write a new function implementing it”, and end up with different functions that accomplish the same task, which are likely to have different bugs. Have every Tesla do this to its own software. Then the virus or program that breaks into some Teslas will likely fail on others.
One reason this is horrible is that you would need an exceptionally high success rate for writing those replacement functions—else this process would introduce lots of mundane bugs, which might well cause crashes of their own. That, or you’d need a very extensive set of unit tests to catch all such bugs—so extensive as to probably eat up most of your engineers’ time writing them. Though perhaps AIs could do that part.

localdeity 23 Oct 2024 18:22 UTC
2 points
0
in reply to: ChristianKl’s comment on: Why I quit effective altruism, and why Timothy Telleen-Lawton is staying (for now)
To me, that will lead to an environment where people think that they are engaging with criticism without having to really engage with the criticism that actually matters.
This is a possible outcome, especially if the above tactic were the only tactic to be employed. That tactic helps reduce ignorance of the “other side” on the issues that get the steelmanning discussion, and hopefully also pushes away low-curiosity tribalistic partisans while retaining members who value deepening understanding and intellectual integrity. There are lots of different ways for things to go wrong, and any complete strategy probably needs to use lots of tactics. Perhaps the most important tactic would be to notice when things are going wrong (ideally early) and adjust what you’re doing, possibly designing new tactics in the process.
Also, in judging a strategy, we should know what resources we assume we have (e.g. “the meetup leader is following the practice we’ve specified and is willing to follow ‘reasonable’ requests or suggestions from us”), and know what threats we’re modeling. In principle, we might sort the dangers by [impact if it happens] x [probability of it happening], enumerate tactics to handle the top several, do some cost-benefit analysis, decide on some practices, and repeat.
If you frame the criticism as having to be about the mission of psychiatry, it’s easy for people to see “Is it ethical to charge poor patients three-digit fees for no-shows?” as off-topic.
My understanding/guess is that “Is it ethical to charge poor patients three-digit fees for no-shows?” is an issue where the psychiatrists know the options and the impacts of the options, and the “likelihood of people actually coming to blows” comes from social signaling things like “If I say I don’t charge them, this shows I’m in a comfortable financial position and that I’m compassionate for poor patients”/”If I say I do charge them, this opens me up to accusations (tinged with social justice advocacy) of heartlessness and greed”. I would guess that many psychiatrists do charge the fees, but would hate being forced to admit it in public. Anyway, the problem here is not that psychiatrists are unaware of information on the issue, so there’d be little point in doing a steelmanning exercise about it.
That said, as you suggest, it is possible that people would spend their time steelmanning unimportant issues (and making ‘criticism’ of the “We need fifty Stalins” type). But if we assume that we have one person who notices there’s an important unaddressed issue, who has at least decent rapport with the meetup leader, then it seems they could ask for that issue to get steelmanned soon. That could cover it. (If we try to address the scenario where no one notices the unaddressed issue, that’s a pretty different problem.)

localdeity 23 Oct 2024 8:02 UTC
9 points
3
on: Why I quit effective altruism, and why Timothy Telleen-Lawton is staying (for now)
I want to register high appreciation of Elizabeth for her efforts and intentions described here. <3
The remainder of this post is speculations about solutions. “If one were to try to fix the problem”, or perhaps “If one were to try to preempt this problem in a fresh community”. I’m agnostic about whether one should try.
Notes on the general problem:
- I suspect lots of our kind of people are not enthusiastic about kicking people out. I think several people have commented, on some cases of seriously bad actors, that it took way too long to actually expel them.
- Therefore, the idea of confronting someone like Jacy and saying “Your arguments are bad, and you seem to be discouraging critical thinking, so we demand you stop it or we’ll kick you out” seems like a non-starter in a few ways.
- I guess one could have lighter policing of the form “When you do somewhat-bad things like that, someone will criticize you for it.” Sort of like Elizabeth arguing against Jacy. In theory, if one threw enough resources at this, one could create an environment where Jacy-types faced consistent mild pushback, which might work to get them to either reform or leave. However, I think this would take a lot more of the required resources (time, emotional effort) than the right people are inclined to give.
  - Those who enjoy winning internet fights… Might be more likely to be Jacy-types in the first place. The intersection of “happy to spend lots of time policing others’ behavior” and “not having what seem like more important things to work on” and “embodies the principles we hope to uphold” might be pretty small. The example that comes to mind is Reddit moderators, who have a reputation for being power-trippers. If the position is unpaid, then it seems logical to expect that result. So I conclude that, to a first approximation, good moderators must be paid.
  - Could LLMs help with this today? (Obviously this would work specifically for online written stuff, not in-person.) Identifying bad comments is one possibility; helping write the criticism is another.
- Beyond that, one could have “passive” practices, things that everyone was in the habit of doing, which would tend to annoy the bad actors while being neutral (or, hopefully, positive) to the good actors.
  - (I’ve heard that the human immune system, in certain circumstances, does basically that: search for antibodies that (a) bind to the bad things and (b) don’t bind to your own healthy cells. Of course, one could say that this is obviously the only sensible thing to do.)
Reading the transcript, my brain generated the idea of having a norm that pushes people to do exercises of the form “Keep your emotions in check as you enumerate the reasons against your favored position, or poke holes in naive arguments for your favored position” (and possibly alternate with arguing for your side, just for balance). In this case, it would be “If you’re advocating that everyone do a thing always, then enumerate exceptions to it”.
Fleshing it out a bit more… If a group has an explicit mission, then it seems like one could periodically have a session where everyone “steelmans” the case against the mission. People sit in a circle, raising their hands (or just speaking up) and volunteering counterarguments, as one person types them down into a document being projected onto a big screen. If someone makes a mockery of a counterargument (“We shouldn’t do this because we enjoy torturing the innocent/are really dumb/subscribe to logical fallacy Y”), then other people gain status by correcting them (“Actually, those who say X more realistically justify it by …”): this demonstrates their intelligence, knowledge, and moral and epistemic strength. Same thing when someone submits a good counterargument: they gain status (“Ooh, that’s a good one”) because it demonstrates those same qualities.
Do this for at least five minutes. After that, pause, and then let people formulate the argument for the mission and attack the counterarguments.

localdeity 23 Oct 2024 5:44 UTC
7 points
0
on: Why I quit effective altruism, and why Timothy Telleen-Lawton is staying (for now)
Issues in transcript labeling (I’m curious how much of it was done by machine):
- After 00:07:55, a line is unattributed to either speaker; looks like it should be Timothy.
- 00:09:43 is attributed to Timothy but I think must be Elizabeth.
- Then the next line is unattributed (should be Timothy).
- After 00:14:00, unattributed (should be Timothy).
- After 00:23:38, unattributed (should be Timothy)
- After 00:32:34, unattributed (probably Elizabeth)

localdeity 10 Oct 2024 3:14 UTC
3 points
−1
in reply to: cubefox’s comment on: Why I’m not a Bayesian
Grammatically, the most obvious interpretation is a universal quantification (i.e. “All men are taller than all women”), which I think is a major reason why such statements so often lead to objections of “But here’s an exception!” Maybe you can tell the audience that they should figure out when to mentally insert ”… on average” or “tend to be”. Though there are also circumstances where one might validly believe that the speaker really means all. I think it’s best to put such qualified language into your statements from the start.

localdeity 9 Oct 2024 22:11 UTC
5 points
2
in reply to: Richard_Ngo’s comment on: Why I’m not a Bayesian
Are you not familiar with the term “vacuously true”? I find this very surprising. People who study math tend to make jokes with it.
The idea is that, if we were to render a statement like “Colorless green ideas sleep furiously” into formal logic, we’d probably take it to mean the universal statement “For all X such that X is a colorless green idea, X sleeps furiously”. A universal statement is logically equivalent to “There don’t exist any counterexamples”, i.e. “There does not exist X such that X is a colorless green idea and X does not sleep furiously”. Which is clearly true, and therefore the universal is equally true.
There is, of course, some ambiguity when rendering English into formal logic. It’s not rare for English speakers to say “if” when they mean “if and only if”, or “or” when they mean “exclusive or”. (And sometimes “Tell me which one”, as in “Did you do A, or B?” “Yes.” “Goddammit.”) Often this doesn’t cause problems, but sometimes it does. (In which case, as I’ve said, the solution is not to give their statement an ambiguous truth value, but rather to ask them to restate it less ambiguously.)
“Dragons are attacking Paris” seems most naturally interpreted as the definite statement “There’s some unspecified number—but since I used the plural, it’s at least 2—of dragons that are attacking Paris”, which would be false. One could also imagine interpreting it as a universal statement “All dragons are currently attacking Paris”, which, as you say, would be vacuously true since there are no dragons. However, in English, the preferred way to say that would be “Dragons attack Paris”, as CBiddulph says. “Dragons are attacking Paris” uses the present progressive tense, while “Dragons attack Paris” uses what is called the “simple present”/”present indefinite” tense. Wiki says:
The simple present is used to refer to an action or event that takes place habitually, to remark habits, facts and general realities, repeated actions or unchanging situations, emotions, and wishes.^[3] Such uses are often accompanied by frequency adverbs and adverbial phrases such as always, sometimes, often, usually, from time to time, rarely, and never.
Examples:
- I always take a shower.
- I never go to the cinema.
- I walk to the pool.
- He writes for a living.
- She understands English.
This contrasts with the present progressive (present continuous), which is used to refer to something taking place at the present moment: I am walking now; He is writing a letter at the moment.
English grammar rules aren’t necessarily universal and unchanging, but they do give at least medium-strength priors on how to interpret a sentence.

localdeity

Per­ils of Gen­er­al­iz­ing from One’s So­cial Group

Perils of Generalizing from One’s Social Group