johnswentworth

Karma: 53,251

johnswentworth Apr 8, 2025, 8:41 PM
8 points
4
in reply to: Jacob Falkovich’s comment on: Navigation by Moonlight
Based on personal experience, the strategy fails to even achieve that goal.
And fun though it is to exchange clever quips, I would much rather have actual answers to useful questions, such as “just how often does anyone at all correctly pick up on the intended subtle signals?”.

johnswentworth Apr 8, 2025, 6:12 PM
5 points
−2
in reply to: jimmy’s comment on: Navigation by Moonlight
Sometimes this is due to the woman in question not recognizing how subtle she’s being, and losing out on a date with a man she’s still interested in.
I would guess that this is approximately 100% of the time in practice, excluding cases where the man doesn’t pick up on the cues but happens to ask her out anyway. Approximately nobody accurately picks up on womens’ subtle cues, including other women (at least that would be my strong guess, and is very cruxy for me here). If the woman just wants a guy who will ask her out, that’s still a perfectly fine utility function, but the cues serve approximately-zero role outside of the woman’s own imagination.
If the typical case was actually to send very clear unambiguous cues which most men (or at least most hot men) actually reliably can pick up on, then I would not call the strategy “completely fucking idiotic”; sending signals which the intended recipient can actually reliably pick up on is a totally reasonable and sensible strategy.
(Of course there’s an obvious alternative hypothesis: most men do pick up on such cues, and I’m overindexing on myself or my friends or something. I am certainly tracking that hypothesis, I am well aware that my brain is not a good model of other humans’ brains, but man it sure sounds like “not noticing womens’ subtle cues” is the near-universal experience, even among other women when people actually try to test that.)

johnswentworth Apr 7, 2025, 8:25 PM
23 points
8
on: Navigation by Moonlight
The yin master finds out from her friend that a certain guy with potential is coming to dinner. She wears a dress with a subtle neckline and arrives early to secure a seat with an empty chair next to it. When the guy arrives, she holds eye contact a fraction longer than usual and invites him to the empty chair by laughing a fraction louder at his silly joke. She leans away when he starts talking sports, but angles slightly towards him when he’s sharing a story of how he got in trouble that one time abroad. The story is good, so she mentions off-handedly how long its been since she saw a good painting and it’s his cue to ask for her number.
This was the one fleshed-out concrete example in the post, and the best part IMO.
I found it quite helpful for visualizing what’s going on in so many women’s imaginations. The post itself delivered value, in that regard.
That said, the post conspicuously avoids asking: how well will this yin strategy actually work? How much will the yin strategy improve this girl’s chance of a date with the guy, compared to (a) doing nothing and acting normally, or (b) directly asking him out? It seems very obvious that the yin stuff will result in a date-probability only marginally higher than doing nothing (I’d say 1-10 percentage points higher at most, if I had to Put A Number On It), and far far lower than if she asks him (I’d say tens of percentage points).
That is what we typically call a completely fucking idiotic strategy. Just do the simple direct thing, chances of success will be far higher.
Now, one could reasonably counter-argue that the yin strategy delivers value somewhere else, besides just e.g. “probability of a date”. Maybe it’s a useful filter for some sort of guy, or maybe she just wants to interact with people this way because she enjoys it? I won’t argue with the utility function; people want what they want. But I will observe that it’s an awfully large trade-off, and I very much doubt that an actual consideration of the tradeoffs under the hypothetical woman’s preferences actually works out in favor of the subtle approach, under almost any real conditions.

johnswentworth Apr 7, 2025, 5:17 PM
29 points
7
on: Coupling for Decouplers
(Context: I’ve been following the series in which this post appeared, and originally posted this comment on the substack version.)
Overcompressed summary of this post: “Look, man, you are not bottlenecked on models of the world, you are bottlenecked on iteration count. You need to just Actually Do The Thing a lot more times; you will get far more mileage out of iterating more than out of modeling stuff.”.
I definitely buy that claim for at least some people, but it seems quite false in general for relationships/dating.
Like, sure, most problems can be solved by iterating infinitely many times. The point of world models is to not need so many damn iterations. And because we live in a very high-dimensional world, naive iteration will often not work at all without a decent world model; one will never try the right things without having some prior idea of where to look.
Example: Aella’s series on how to be good in bed. I was solidly in the audience for that post: I’d previously spent plenty of iterations becoming better in bed, ended up with solid mechanics, but consistently delivering great orgasms does not translate to one’s partner wanting much sex. Another decade of iteration would not have fixed that problem; I would not have tried the right things, my partner would not have given the right feedback (indeed, much of her feedback was in exactly the wrong direction). Aella pointed in the right vague direction, and exploring in that vague direction worked within only a few iterations. That’s the value of models: they steer the search so that one needs fewer iterations.
That’s the point of all the blog posts. That’s where the value is, when blog posts are delivering value. And that’s what’s been frustratingly missing from this series so far. (Most of my value of this series has been from frustratedly noticing the ways in which it fails to deliver, and thereby better understanding what I wish it would deliver!) No, I don’t expect to e.g. need 0 iterations after reading, but I want to at least decrease the number of iterations.
And in regards to “I don’t think you know what you want in dating”… the iteration problem still applies there! It is so much easier to figure out what I want, with far fewer iterations, when I have better background models of what people typically want. Yes, there’s a necessary skill of not shoving yourself into a box someone else drew, but the box can still be extremely valuable as evidence of the vague direction in which your own wants might be located.

johnswentworth Apr 7, 2025, 4:11 AM
LW: 12 AF: 8
0
AF
in reply to: Joe Carlsmith’s comment on: AI for AI safety
(This comment is not about the parts which most centrally felt anchored on social reality; see other reply for that. This one is a somewhat-tangential but interesting mini-essay on ontological choices.)
The first major ontological choices were introduced in the previous essay:
1. Thinking of “capability” as a continuous 1-dimensional property of AI
2. Introducing the “capability frontier” as the highest capability level the actor developed/deployed so far
3. Introducing the “safety range” as the highest capability level the actor can safely deploy
4. Introducing three “security factors”:
  1. Making the safety range (the happy line) go up
  2. Making the capability frontier (the dangerous line) not go up
  3. Keeping track of where those lines are.
The first choice, treatment of “capability level” as 1-dimensional, is obviously an oversimplification, but a reasonable conceit for a toy model (so long as we remember that it is toy, and treat it appropriately). Given that we treat capability level as 1-dimensional, the notion of “capability frontier” for any given actor immediately follows, and needs no further justification.
The notion of “safety range” is a little more dubious. Safety of an AI obviously depends on a lot of factors besides just the AI’s capability. So, there’s a potentially very big difference between e.g. the most capable AI a company “could” safely deploy if the company did everything right based on everyone’s current best understanding (which no company of more than ~10 people has or ever will do in a novel field), vs the most capable AI the company could safely deploy under realistic assumptions about the company’s own internal human coordination capabilities, vs the most capable AI the company can actually-in-real-life aim for and actually-in-real-life not end up dead.
… but let’s take a raincheck on clarifying the “safety range” concept and move on.
The safety factors are a much more dubious choice of ontology. Some of the dubiousness:
- If we’re making the happy line go up (“safety progress”): who’s happy line? Different actors have different lines. If we’re making the danger line not go up (“capability restraint”): again, who’s danger line? Different actors also have different danger lines.
  - This is important, because humanity’s survival depends on everybody else’s happy and danger lines, not just one actor’s!
- “Safety progress” inherits all the ontological dubiousness of the “safety range”.
- If we’re keeping track of where the lines are (“risk evaluation”): who is keeping track? Who is doing the analysis, who is consuming it, how does the information get to relevant decision makers, and why do they make their decisions on the basis of that information?
- Why factor apart the levels of the danger and happy lines? These are very much not independent, so it’s unclear why it makes sense to think of their levels separately, rather than e.g. looking at their average and difference as the two degrees of freedom, or their average and difference in log space, or the danger line level and the difference, or [...]. There’s a lot of ways to parameterize two degrees of freedom, and it’s not clear why this parameterization would make more sense than some other.
- On the other hand, factoring apart “risk evaluation” from “safety progress” and “capabilities restraint” does seem like an ontologically reasonable choice: it’s the standard factorization of instrumental from epistemic. That standard choice is not always the right way to factor things, but it’s at least a choice which has “low burden of proof” in some sense.
What would it look like to justify these ontological choices? In general, ontological justification involves pointing to some kind of pattern in the territory—in this case, either the “territory” of future AI, or the “territory” of AI safety strategy space. For instance, in a very broad class of problems, one can factor apart the epistemic and instrumental aspects of the problem, and resolve all the epistemic parts in a manner totally agnostic to the instrumental parts. That’s a pattern in the “territory” of strategy spaces, and that pattern justifies the ontological choice of factoring apart instrumental and epistemic components of a problem.
If one could e.g. argue that the safety range and capability frontier are mostly independent, or that most interventions impact the trajectory of only one of the two, then that would be an ontological justification for factoring the two apart. (Seems false.)
(To be clear: people very often have good intuitions about ontological choices, but don’t know how to justify them! I am definitely not saying that one must always explicitly defend ontological choices, or anything like that. But one should, if asked and given time to consider, be able to look at an ontological choice and say what underlying pattern makes that ontological choice sensible.)

johnswentworth Apr 7, 2025, 4:08 AM
LW: 5 AF: 4
−1
AF
in reply to: Joe Carlsmith’s comment on: AI for AI safety
The last section felt like it lost contact most severely. It says
What are the main objections to AI for AI safety?
It notably does not say “What are the main ways AI for AI safety might fail?” or “What are the main uncertainties?” or “What are the main bottlenecks to success of AI for AI safety?”. It’s worded in terms of “objections”, and implicitly, it seems we’re talking about objections which people make in the current discourse. And looking at the classification in that section (“evaluation failures, differential sabotage, dangerous rogue options”) it indeed sounds more like a classification of objections in the current discourse, as opposed to a classification of object-level failure modes from a less-social-reality-loaded distribution of failures.
I do also think the frame in the earlier part of the essay is pretty dubious in some places, but that feels more like object-level ontological troubles and less like it’s anchoring too much on social reality. I ended up writing a mini-essay on that which I’ll drop in a separate reply.

johnswentworth Apr 6, 2025, 5:23 PM
3 points
0
on: Thoughts on Creating a Good Language
no synonyms
[...]
Use compound words.
These two goals conflict. When compounding is common, there will inevitably be multiple reasonable ways to describe the same concept as a compound word. I think you probably want flexible compounding more than a lack of synonyms.

johnswentworth Apr 4, 2025, 7:54 PM
LW: 18 AF: 9
0
AF
on: AI for AI safety
When this post first came out, it annoyed me. I got a very strong feeling of “fake thinking”, fake ontology, etc. And that feeling annoyed me a lot more than usual, because Joe is the person who wrote the (excellent) post on “fake vs real thinking”. But at the time, I did not immediately come up with a short explanation for where that feeling came from.
I think I can now explain it, after seeing this line from kave’s comment on this post:
Your taxonomies of the space of worries and orientations to this question are really good...
That’s exactly it. The taxonomies are taxonomies of the space of worries and orientations. In other words, the post presents a good ontology of the discourse on “AI for AI safety”. What it does not present is an ontology natural to the actual real-world challenges of AI for AI safety.
Unpacking that a bit: insofar as an ontology (or taxonomy) is “good”, it reflects some underlying pattern in the world, and it’s useful to ask what that underlying pattern is. For instance, it does intuitively seem like most objections to “AI for AI safety” I hear these days cluster reasonably well into “Evaluation failures, Differential sabotage, Dangerous rogue options”. Insofar as the discourse really does cluster that way, that’s a real pattern in the world-of-discourse, and those categories are useful for modelling the discourse. But the patterns in the discourse mostly reflect social dynamics; they are only loosely coupled to the patterns which will actually arise in future AIs, or in the space of strategies for dealing with future AIs. Thus the feeling of “fakeness”: it feels like this post is modeling the current discourse, rather than modeling the actual physical future AIs.
… and to be clear, that’s not strictly a bad thing. Modeling the discourse might actually be the right move, insofar as one’s main goal is to e.g. facilitate communication. That communication just won’t be very tightly coupled to future physical reality.

johnswentworth Apr 3, 2025, 4:58 PM
22 points
2
on: How To Believe False Things
I have a similar story. When I was very young, my mother was the primary breadwinner of the household, and put both herself and my father through law school. Growing up, it was always just kind of assumed that my sister would have to get a real job making actual money, same as my brother and I; a degree in underwater basket weaving would have required some serious justification. (She ended up going to dental school and also getting a PhD working with epigenomic data.)
I didn’t realize on a gut level that this wasn’t the norm until shortly after high school. I was hanging out with two female friends and one of them said “man, I really need more money”. I replied “sounds like you need to get a job”. The friend laughed and said “oh, I was thinking I need to get a boyfriend”, and then the other friend also laughed and said she was also thinking the boyfriend thing.
… so that was quite a shock to my worldview.

johnswentworth Apr 2, 2025, 3:46 PM
5 points
1
on: Building Communities Beyond the Bay
Not important, but: I clicked on this post expecting an essay about building physical islands outside of San Francisco bay.

johnswentworth Apr 1, 2025, 4:39 PM
6 points
0
in reply to: Lucius Bushnaq’s comment on: johnswentworth’s Shortform
This comment gave me the information I’m looking for, so I don’t want to keep dragging people through it. Please don’t feel obligated to reply further!
That said, I did quickly look up some data on this bit:
But remember that you already conditioned on ‘married couples without kids’. My guess would be that in the subset of man-woman married couples without kids, the man being the exclusive breadwinner is a lot less common than in the set of all man-woman married couples.
… so I figured I’d drop it in the thread.
When interpreting these numbers, bear in mind that many couples with no kids probably intend to have kids in the not-too-distant future, so the discrepancy shown between “no children” and 1+ children is probably somewhat smaller than the underlying discrepancy of interest (which pushes marginally more in favor of Lucius’ guess).

johnswentworth Apr 1, 2025, 3:57 PM
2 points
0
in reply to: Lucius Bushnaq’s comment on: johnswentworth’s Shortform
Big thank you for responding, this was very helpful.

johnswentworth Mar 24, 2025, 3:56 PM
6 points
0
in reply to: Zack_M_Davis’s comment on: johnswentworth’s Shortform
That is useful, thanks.
Any suggestions for how I can better ask the question to get useful answers without apparently triggering so many people so much? In particular, if the answer is in fact “most men would be happier single but are ideologically attached to believing in love”, then I want to be able to update accordingly. And if the answer is not that, then I want to update that most men would not be happier single. With the current discussion, most of what I’ve learned is that lots of people are triggered by the question, but that doesn’t really tell me much about the underlying reality.

johnswentworth Mar 24, 2025, 1:14 AM
1 point
−18
in reply to: johnswentworth’s comment on: johnswentworth’s Shortform
Update 3 days later: apparently most people disagree strongly with
Their romantic partner offering lots of value in other ways. I’m skeptical of this one because female partners are typically notoriously high maintenance in money, attention, and emotional labor. Sure, she might be great in a lot of ways, but it’s hard for that to add up enough to outweigh the usual costs.
Most people in the comments so far emphasize some kind of mysterious “relationship stuff” as upside, but my actual main update here is that most commenters probably think the typical costs are far far lower than I imagined? Unsure, maybe the “relationship stuff” is really ridiculously high value.
So I guess it’s time to get more concrete about the costs I had in mind:
- A quick google search says the male is primary or exclusive breadwinner in a majority of married couples. Ass-pull number: the monetary costs alone are probably ~50% higher living costs. (Not a factor of two higher, because the living costs of two people living together are much less than double the living costs of one person. Also I’m generally considering the no-kids case here; I don’t feel as confused about couples with kids.)
- I was picturing an anxious attachment style as the typical female case (without kids). That’s unpleasant on a day-to-day basis to begin with, and I expect a lack of sex tends to make it a lot worse.
- Eyeballing Aella’s relationship survey data, a bit less than a third of respondents in 10-year relationships reported fighting multiple times a month or more. That was somewhat-but-not-dramatically less than I previously pictured. Frequent fighting is very prototypically the sort of thing I would expect to wipe out more-than-all of the value of a relationship, and I expect it to be disproportionately bad in relationships with little sex.
- Less legibly… conventional wisdom sure sounds like most married men find their wife net-stressful and unpleasant to be around a substantial portion of the time, especially in the unpleasant part of the hormonal cycle, and especially especially if they’re not having much sex. For instance, there’s a classic joke about a store salesman upselling a guy a truck, after upselling him a boat, after upselling him a tackle box, after [...] and the punchline is “No, he wasn’t looking for a fishing rod. He came in looking for tampons, and I told him ‘dude, your weekend is shot, you should go fishing!’”.
(One thing to emphasize in these: sex isn’t just a major value prop in its own right, I also expect that lots of the main costs of a relationship from the man’s perspective are mitigated a lot by sex. Like, the sex makes the female partner behave less unpleasantly for a while.)
So, next question for people who had useful responses (especially @Lucius Bushnaq and @yams): do you think the mysterious relationship stuff outweighs those kinds of costs easily in the typical case, or do you imagine the costs in the typical case are not all that high?

johnswentworth Mar 24, 2025, 12:18 AM
4 points
3
in reply to: Elizabeth’s comment on: johnswentworth’s Shortform
men are the ones who die sooner if divorced, which suggests
Causality dubious, seems much more likely on priors that men who divorced are disproportionately those with Shit Going On in their lives. That said, it is pretty plausible on priors that they’re getting a lot out of marriage.

johnswentworth Mar 21, 2025, 6:16 PM
4 points
0
in reply to: yams’s comment on: johnswentworth’s Shortform
I think you’ll get the most satisfying answer to your actual question by having a long chat with one of your asexual friends (as something like a control group, since the value of sex to them is always 0 anyway, so whatever their cause is for having romantic relationships is probably the kind of thing that you’re looking for here).
That’s an excellent suggestion, thanks.

johnswentworth Mar 21, 2025, 4:02 PM
3 points
2
in reply to: Lucius Bushnaq’s comment on: johnswentworth’s Shortform
Yes, the question is what value-proposition accounts for the romantic or general relationship satisfaction.

johnswentworth Mar 21, 2025, 4:39 AM
11 points
−31
on: johnswentworth’s Shortform
Here’s a place where I feel like my models of romantic relationships are missing something, and I’d be interested to hear peoples’ takes on what it might be.
Background claim: a majority of long-term monogamous, hetero relationships are sexually unsatisfying for the man after a decade or so. Evidence: Aella’s data here and here are the most legible sources I have on hand; they tell a pretty clear story where sexual satisfaction is basically binary, and a bit more than half of men are unsatisfied in relationships of 10 years (and it keeps getting worse from there). This also fits with my general models of mating markets: women usually find the large majority of men sexually unattractive, most women eventually settle on a guy they don’t find all that sexually attractive, so it should not be surprising if that relationship ends up with very little sex after a few years.
What doesn’t make sense under my current models is why so many of these relationships persist. Why don’t the men in question just leave? Obviously they might not have better relationship prospects, but they could just not have any relationship. The central question which my models don’t have a compelling answer to is: what is making these relationships net positive value for the men, relative to not having a romantic relationship at all?
Some obvious candidate answers:
- Kids. This one makes sense for those raising kids, but what about everyone else? Especially as fertility goes down.
- The wide tail. There’s plenty of cases which make sense which are individually unusual—e.g. my own parents are business partners. Maybe in aggregate all these unusual cases account for the bulk.
- Loneliness. Maybe most of these guys have no one else close in their life. In this case, they’d plausibly be better off if they took the effort they invested in their romantic life and redirected to friendships (probably mostly with other guys), but there’s a lot of activation energy blocking that change.
- Their romantic partner offering lots of value in other ways. I’m skeptical of this one because female partners are typically notoriously high maintenance in money, attention, and emotional labor. Sure, she might be great in a lot of ways, but it’s hard for that to add up enough to outweigh the usual costs.
- Wanting a dependent. Lots of men are pretty insecure, and having a dependent to provide for makes them feel better about themselves. This also flips the previous objection: high maintenance can be a plus if it makes a guy feel wanted/useful/valuable.
- Social pressure/commitment/etc making the man stick around even though the relationship is not net positive for him.
- The couple are de-facto close mostly-platonic friends, and the man wants to keep that friendship.
I’m interested in both actual data and anecdata. What am I missing here? What available evidence points strongly to some of these over others?
Edit-to-add: apparently lots of people are disagreeing with this, but I don’t know what specifically you all are disagreeing with, it would be much more helpful to at least highlight some specific sentence or leave a comment or something.

johnswentworth Mar 20, 2025, 5:10 PM
6 points
1
on: How far along Metr’s law can AI start automating or helping with alignment research?
Fun point! If I just pull some numbers out of my ass and naively plug them into the equation:
- Superalignment-relevant research tasks take time on the order of a year to a decade.
- Taking 1 hr as the current task-time models can handle, we’d need ~13-16 doublings.
- … so on METR’s model, that’s about 8-10 years from now.
Of course that’s assuming the model holds. A useful heuristic for trends which have just been published for the first time is that they break down immediately after publication. If it holds up for another year or two, then the model will look a lot more plausible.

johnswentworth Mar 20, 2025, 4:45 PM
6 points
2
in reply to: Alexander Gietelink Oldenziel’s comment on: Why Are The Human Sciences Hard? Two New Hypotheses
Although I wouldn’t dispute the stats that you are citing here, John, I would guess these might be downstream from above difficulties.
I think that’s the right counterargument to make (kudos :) ). Building on it, I’d say: ok, the causal arrows go both ways here. The field draws less impressive people, who do kinda junk research, which draws less impressive people. So the different sciences end up in different equilibria, with different levels of competence among the scientists. But then, what determines the equilibrium? Why did physics end up with more competent people doing more competent work, and psychology with less competent people doing less competent work, rather than vice-versa? Unless we want to claim that it was luck of the draw, there has to be something endogenous to the underlying territories of physics and psychology which cause their sciences to end up in these different equilibria.
Once the question is framed that way, it suggests different answers.
One could maybe tell a story like “psychology is more complex than physics, so physics had more impressive work earlier on, which drew in smarter people...”. That could explain the different equilibria. But also, it feels like a just-so story; one could just as easily argue that smarter people tend to be drawn to more complex systems, where they can properly show off their merit.
Explanations like noisy data or difficulties with controls seem similar. Like, any explanation of the form “psychology is harder for reason X, so physics had more impressive work earlier on, which drew in smarter people...” feels like a just-so story; it seems at least as plausible that more competent people will be drawn to more difficult problems. (Note that both the hypotheses put forward in the OP are of this form, so this is also a response to the OP.)
Feedback loops look a little more plausible as an explanation: “insofar as it’s harder to get clear feedback on models in psychology compared to physics, it’s easier for bullshit to thrive in psychology, so smarter people go to physics where success is relatively more a function of smarts rather than academic politics, …”. I don’t really buy that explanation, since getting feedback on models seems-to-me comparably difficult in physics and in psychology; it’s a core hard part of any science. But it’s at least plausible.
The infiltration of “social reality” is another plausible explanation, which I personally find much more probable. The model would be roughly: “insofar as psychology is largely about modeling social reality and adjacent topics, psychology itself becomes a relevant battleground for social-reality-level competitions, and also people predisposed to thinking in social reality will find psychology more appealing than physics. Alas, both focus on social-reality-level competitions and predisposition for thinking in social reality are extremely strong negative predictors of one’s competence as a scientist, so psychology ends up with a lot more incompetent people than physics, …”. I probably wouldn’t endorse that exact model as worded, but I’d put a fair bit of probability on something in roughly that ballpark.