Benquo

• Benquo 10 Feb 2025 3:36 UTC

  2 points

  0

  in reply to: Hzn’s comment on: Park­in­son’s Law and the Ide­ol­ogy of Statistics

  I built the explanatory model based on my experience employed by and reading about other vaguely analogous institutions, but an acquaintance who’d previously worked at the World Bank said it seemed like an accurate characterization of that institution as well.

• Benquo 31 Jan 2025 17:32 UTC

  2 points

  0

  in reply to: cousin_it’s comment on: Pre­da­tion as Pay­ment for Criticism

  The post describes how predation creates a specific gradient favoring better modeling of predator behavior. While fact that most predated species don’t develop high intelligence is Bayesian evidence against this explanation, it’s very weak counterevidence because general self-aware intelligence is a very narrow target. More importantly, why would sexual selection specifically target intelligence rather than any other trait?

  Looking at peacocks, we can see what appears to be an initial predation-driven selection for looking like they had big intimidating eyes on their backs (similar to butterflies), followed by sexual selection amplifying along roughly that same gradient direction.

Pre­da­tion as Pay­ment for Criticism

• Benquo 20 Jan 2025 1:37 UTC

  2 points

  0

  in reply to: Viliam’s comment on: Guilt, Shame, and Depravity

  Contempt of court penalties for noncompliance with an investigative process is a mainstream example of 1.

  Burning Man has some aspects of the second, as do some camping trips, or simply living in a relatively harsh climate. Compare measured levels of corruption in southern vs northern Europe, for instance. When modern democracies fight big wars, the first year involves learning which parts of their warfighting institutions are corrupt and incompetent, & repairing or replacing them.

• Benquo 16 Jan 2025 2:22 UTC

  8 points

  1

  on: Rough Sketch for Product to En­hance Ci­ti­zen Par­ti­ci­pa­tion in Politics

  Your proposal is well-structured and interesting but has a fundamental flaw that needs to be addressed. Interest keyword-based filtering will primarily encourage politics-as-identity, which is actively harmful—it directs attention towards zero-sum thinking and performative identities, rather than creative problem solving. As Bryan Caplan demonstrates in The Myth of the Rational Voter, people already tend to vote to express identities and affiliations rather than to achieve better outcomes. We shouldn’t build tools that further entrench this destructive pattern.

  Instead, imagine a tool that:

  1. Has users journal daily about their life—activities, hopes, problems, and worries

  2. Uses AI to identify where their constraints are plausibly caused by or could be alleviated by government action, especially local government

  3. Maps them to specific opportunities for formal recourse, with guidance on process, likely outcomes, and practical assistance (like drafting letters or legal documents)

  4. For issues requiring collective action, connects users facing similar constraints and helps coordinate through mechanisms like dominant assurance contracts where appropriate

  This approach would ground political participation in the solving of one’s own problems rather than identity expression. While technically more challenging to implement than interest-based filtering, it would generate higher-quality engagement that expands our collective problem-solving capacity rather than just reallocating political power between existing interest groups.

  The patterns emerging from aggregated user experiences would naturally reveal systemic issues and preventive opportunities, especially in how regulations and policies interact to shape people’s choices and planning horizons. While building reliable AI judgment about political causation is challenging, it’s better to attempt something hard that would be beneficial if feasible, than to facilitate the destructive forces of identity-based politics simply because they’re easier to implement.

LLMs for lan­guage learning

• Benquo 13 Jan 2025 1:56 UTC

  4 points

  4

  in reply to: Hzn’s comment on: Park­in­son’s Law and the Ide­ol­ogy of Statistics

  I agree that even if the book turned out to be entirely accurate we should not assume that this case is representative of the average development project, but we could still learn from it. Many hours from highly trained and well-paid people are allocated to planning and evaluating such projects, which expenditure is ostensibly to ensure quality. Even looking at worst cases helps us see what sort of quality is or is not being ensured.

• Benquo 12 Jan 2025 1:49 UTC

  8 points

  0

  in reply to: kave’s comment on: Park­in­son’s Law and the Ide­ol­ogy of Statistics

  Wow, thanks for doing the legwork on this—seems like quite possibly I’m analyzing fiction? Annoying if true.

  Google’s AI response to my search for the Thaba-Tseka Development Project says:

  According to available World Bank documentation, the “Thaba-Tseka development project” is primarily referenced within the context of the “Lesotho Integrated Transport, Trade and Logistics Project,” which focuses on improving the road corridor connecting Katse to Thaba-Tseka, aiming to enhance regional connectivity and reduce trade costs at Lesotho’s borders with South Africa; key documents to reference would be those related to this project, particularly those detailing the road infrastructure development component between Katse and Thaba-Tseka.

  Key points about the documentation:

  • Project Title: “Lesotho Integrated Transport, Trade and Logistics Project”

  • Focus Area: Upgrading the Katse to Thaba-Tseka road corridor

  • Objectives: Improve climate resilient regional connectivity, reduce trade costs at Lesotho’s borders

  • Relevant documents to explore: Project Appraisal Documents, Procurement documents related to road construction and improvement on the Katse-Thaba-Tseka stretch

  There’s a good chance this is an AI hallucination, though; a cursory search of the main documents didn’t yield any references to a “Thaba-Tseka development project,” or the wood or ponies. I’m not familiar with World Bank documentation, though, and likely the right followup would involve looking at exactly what’s cited in the book.

  However, the other lead funder, the Canadian International Development Agency, does seem to have at least one publicly referenced document about a “Thaba-Tseka rural development program”: Evaluation, the Kingdom of Lesotho rural development : evaluation design for phase 1, the Thaba Tseka project

• Benquo 12 Jan 2025 1:15 UTC

  6 points

  0

  in reply to: jimmy’s comment on: Prefer­ence Inversion

  Initially, you argued that societal pressure often reflects genuine wisdom, using examples where a ‘society who aggressively shames overconsumption of sweets’ might be wiser than a child’s raw preferences. You suggested that what I was calling ‘intrinsic preferences’ might just be ‘shallow preferences’ that hadn’t yet been trained to reflect reality.

  Now you’re making a different and more sophisticated argument—that the whole framework of ‘intrinsic’ versus ‘external’ preferences is problematic because preferences necessarily develop within and respond to reality, including social reality. While this is an interesting perspective that deserves consideration, it seems substantially different from your initial defense of social restrictions as transmitting wisdom.

  There’s also an important point about my own position that I should clarify. When I said ‘generally, upon reflection, people would prefer to satisfy their and others’ preferences as calculated prior to such influences,′ I wasn’t making a claim about how often admonitions reflect preference inversions. Rather, I was suggesting that if people were to reflect explicitly on cases of preference inversion, they typically wouldn’t want those inverted preferences to count; they would recognize these as preferences shaped by forces systematically opposed to their interests.

  This connects to what I see as the core distinction: I’m not just talking about external influences or errors in the transmission of wisdom. I’m specifically pointing to cases where restrictions are moralized for the purpose of restriction itself—where the system is systematically deprecating the evolutionarily fit preferences of the person being restricted. This isn’t just clumsy teaching or social pressure—it’s adversarial. The system works by first making people feel guilty about their natural inclinations, then betting that they won’t fully succeed at suppressing those inclinations despite earnestly trying to adopt the system’s restrictions.

  Consider the survival of variants of Christianity that ‘do poorly’ at helping people develop healthy attitudes toward sexuality. Their persistence suggests this poor performance is actually functional—they are able to exploit their members precisely because they create a system where most people must be ‘bad’ by design, where hypocrisy isn’t a bug but a feature. When dessert companies can successfully market their products as ‘sinfully delicious,’ they’re exploiting a system of moral restrictions that creates the very compulsive relationship to sweets it claims to prevent.

• Benquo 11 Jan 2025 2:38 UTC

  2 points

  0

  in reply to: CstineSublime’s comment on: Guilt, Shame, and Depravity

  Different example—I said “instead”

  If you look back, you’ll see I was specifically responding to the hypothetical scenario about public admission in that comment. For your points about private shame, you might want to check my other comment replying to you where I addressed how internal shame and self-image maintenance connect to social dynamics.

  I notice you’re attributing positions to me that I haven’t taken and expressing confusion about points I’ve already addressed in detail. It would be helpful if you could engage more carefully with what I’ve carefully written.

  so if the musician openly admits and apologize for only being average they are ashamed because they are afraid of the reaction of the fan who clearly loved their performance (not their failure to abstain from what they believe is the cause of their average performance?)

  You’re introducing new elements that weren’t in your original scenario. But more importantly: you described the show as “a hit” where “everyone loves them.” Calling this performance “only average” isn’t accurately revealing adverse information—it’s a lie.

  but if they don’t mention it to anyone (therefore are committing neither a dominance nor submission gesture) they are also ashamed?

  In my other reply to you, I explained how private shame often involves maintaining conflicting mental models—one that enables confident performance and another that tracks specific flaws for improvement. Even when no one would directly know or care about staying up late drinking, the performer may feel shame because they’ve invested in an identity as a “professional musician” or “disciplined performer”—an identity that others care about and grant certain privileges to. The shame comes from violating the requirements of this identity, which serves as a proxy for social approval and professional opportunities. This creates internal pressure toward shame even without a specific idea of someone else who would directly condemn the behavior or trait in question.

  Are you telling me there is no conceivable circumstance where any human being feels shame for something which is totally alone, none at all?

  What I’m suggesting is that shame inherently involves at least a tacit social component—some imagined perspective by which we are condemned. This is consistent with Smith’s and Hume’s moral sentiments theory, where moral judgments fundamentally involve taking up imagined perspectives of others. This doesn’t mean the shame isn’t genuinely felt or that any specific others would actually condemn us. But in my experience people can frequently unravel particular cases of such shame by honestly examining what specific others would actually think if they knew, which is some experimental validation for this view.

• Benquo 11 Jan 2025 1:25 UTC

  2 points

  0

  in reply to: CstineSublime’s comment on: Guilt, Shame, and Depravity

  Except frequently I think people who are ashamed don’t expect this.

  That’s why I distinguished explicitly between shame and depravity in the OP.

• Benquo 11 Jan 2025 1:23 UTC

  2 points

  0

  in reply to: CstineSublime’s comment on: Guilt, Shame, and Depravity

  In this example?

  Except frequently I think people who are ashamed don’t expect this. Imagine that instead of concealing they openly admit and apologize for being only average: then what? Aren’t they still ashamed?

• Benquo 10 Jan 2025 17:24 UTC

  6 points

  2

  in reply to: Chris_Leong’s comment on: Dis­cur­sive War­fare and Fac­tion Formation

  I’m thinking of cases like Eliezer’s Politics is the Mind-Killer, which makes the relatively narrow claim that politically loaded examples are bad examples for illustrating principles of rationality in the context of learning and teaching those principles, so they should be avoided when a less politicized alternative is available. I think this falsely assumes that it’s feasible under current circumstances for some facts to be apolitical in the absence of an active, political defense of the possibility of apolitical speech. But that’s a basically reasonable and sane mistake to make. Then I see LessWrongers proceed as though Politics is the Mind-Killer established canonically that it is bad to mention when someone is saying or doing something politically loaded or discuss recognized-as-political precedents, which interferes with the sort of defense that Politics is the Mind-Killer implicitly assumed was a solved problem.

  Or how Eliezer both explicitly wrote at length against treating intellectual authorities as specially entitled to opinions AND played with themes of being an incomprehensibly powerful optimization process, but the LessWrong community ended up crystallizing around an exaggerated version of the latter while mostly ignoring his explicit warnings against authority-based reasoning. Eliezer’s personally commented on this (higher-context link that may take longer to load):

  “How dare you think that you’re better at meta-rationality than Eliezer Yudkowsky, do you think you’re special”—is somebody trolling? Have they never read anything I’ve written in my entire life? Do they have no sense, even, of irony? Yeah, sure, it’s harder to be better at some things than me, sure, somebody might be skeptical about that, but then you ask for evidence or say “Good luck proving that to us all eventually!” You don’t be like, “Do you think you’re special?” What kind of bystander-killing argumentative superweapon is that? What else would it prove?

  I really don’t know how I could make this any clearer. I wrote a small book whose second half was about not doing exactly this. I am left with a sense that I really went to some lengths to prevent this, I did what society demands of a person plus over 10,000% (most people never write any extended arguments against bad epistemology at all, and society doesn’t hold that against them), I was not subtle. At some point I have to acknowledge that other human beings are their own people and I cannot control everything they do—and I hope that others will also acknowledge that I cannot avert all the wrong thoughts that other people think, even if I try, because I sure did try. A lot. Over many years. Aimed at that specific exact way of thinking. People have their own wills, they are not my puppets, they are still not my puppets even if they have read some blog posts of mine or heard summaries from somebody else who once did; I have put in at least one hundred times the amount of effort that would be required, if any effort were required at all, to wash my hands of this way of thinking.

  Or how Eliezer wrote about how modern knowledge work has become harmfully disembodied and dissociated from physical reality—going into detail about how running from a tiger engages your whole sensorimotor system in a way that staring at a computer screen doesn’t—but lots of Lesswrongers seem to endorse and even celebrate this very dissociation from physical reality in practice.

• Benquo 10 Jan 2025 16:43 UTC

  2 points

  0

  in reply to: sunwillrise’s comment on: Guilt, Shame, and Depravity

  I agree.

  When applied to object-level behavior like stealing cookies, this kind of norm internalization is ethically neutral. But when applied to protocols and coordination mechanisms, this becomes part of how shame-based coordination infiltrates and subverts communities doing something more interesting—people who recognize and try to leave bad communities end up recreating those same dysfunctional behaviors in the better communities they seek out.
  
  In my reply to CstineSublime on pecking orders I explored how this works through specific social mechanisms like using self-deprecation to derail accountability.

• Benquo 10 Jan 2025 15:56 UTC

  2 points

  0

  in reply to: CstineSublime’s comment on: Guilt, Shame, and Depravity

  Admitting and apologizing for being ‘only average’ often functions as a submission move in dominance hierarchies, i.e. pecking orders.

  This move derails attempts to enact more naïve, descriptive-language accountability. When someone has a specific grievance, it corresponds to a claim about the relation between facts and commitments that can be evaluated as true or false. Responding with self-deprecation transforms their concrete complaint into a mere opportunity to either accept or reject the display of submission. This disrupts the sort of language in which object-level accounting can happen, since the original specific issues are neither addressed nor refuted. Rather, they are displaced by the lower-dimensional social dynamics of dominance and submission.
  
  So viewed systemically, such moves are part of a distributed strategy by which pecking orders disrupt and displace descriptive language communities by coordinating to invalidate them. And viewed locally, they erase the specific grievance from common knowledge, preserving the motivating shame.

• Benquo 10 Jan 2025 15:39 UTC

  2 points

  −1

  in reply to: CstineSublime’s comment on: Guilt, Shame, and Depravity

  We conceal some facts about ourselves from ourselves to maintain a self-image because such self-images affect how we present ourselves to others and thus what we can be socially entitled to. This is similar to what psychologist Carol Dweck called a “fixed mindset,” in contrast with a “growth mindset” where the self-image more explicitly includes the possibility of intentional improvement.

  In the singer-songwriter example, creating a good vibe with the audience generally involves projecting confidence. This confidence can connect to an identity as a competent performer, which maintains entitlement to the audience’s approval as well as other perks like booking future shows and charging higher rates. We might think of the performer as implicitly reasoning, “I must have audience approval in order to maintain my identity. I get audience approval by being a good performer. Therefore I must be a good performer. Good performers perform flawlessly. Therefore I must have performed flawlessly. Staying out late would cause flaws in my performance. Therefore I must not have stayed out late.”

  Meanwhile, improving as a performer requires honestly evaluating weaknesses in one’s performance—noticing timing issues, pitch problems, or moments where energy flagged. This evaluation process works best with immediate, specific feedback while memories are fresh. Or, in the specific example you gave, the performer’s process of improvement needs to include the specific factual memory that they stayed out late, which likely impaired their performance.

  When the good vibe with the audience is based on a rigidly maintained self-image, this creates an internal conflict: The same performance needs to be confidently good for maintaining entitlement and specifically flawed to enable improvement. This conflict creates pressure toward shame—the performer must maintain a persona that cannot acknowledge certain facts, while those facts are still actively used to make decisions.
  
  Some other prior work on this topic:

  • Robin Hanson:

    • A Theory of Identity

  • Paul Christiano:

    • If we can’t lie to others, we will lie to ourselves

    • Epistemic incentives and sluggish updating

• Benquo 9 Jan 2025 20:36 UTC

  2 points

  0

  in reply to: jimmy’s comment on: Prefer­ence Inversion

  Seems like we’ve now established that we largely agree on the explicit propositions we’ve stated all through this thread. Given that, your initial response feels to me like a bit of a non-sequitur.

  As I understand it, your response argued against a universal claim that social pressure always inverts genuine preferences, while I had explicitly made the narrower claim that this sometimes happens and is worth watching out for. Does that seem like a fair characterization? If so, can you help me understand why your initial response felt important and relevant to you in context?

• Benquo 9 Jan 2025 16:59 UTC

  10 points

  0

  in reply to: Martin Randall’s comment on: Prefer­ence Inversion

  Many historical battles have a large component well modeled as a game of chicken, where whichever side’s morale breaks first loses. You can get a locally cheap boost to morale if your soldiers have internalized a cultural imperative to seek death in honorable combat, because they’ll be less deterrable. There’s plenty of credible literary evidence that many soldiers in cultures connected to ours were so acculturated. I am not claiming that it is a human universal, merely that it happens often enough to be an important example of preference inversion.

Dis­cur­sive War­fare and Fac­tion Formation

• Benquo 9 Jan 2025 3:06 UTC

  2 points

  0

  in reply to: Martin Randall’s comment on: Prefer­ence Inversion

  This seems like a reasonable argument for some premodern fighting. I meant mainly the way of fighting developed in the Napoleonic wars, the American Civil War, and especially WWI. There’s a bit in Mein Kampf about how WWI was a major transition in character for Hitler because he switched from fearing danger to intending to move towards danger. Worth reading carefully. The sort of fatalistic stories where people with foreknowledge of their doom keep moving towards it in old warrior-culture texts like the Eddas also seem relevant here. These very much do not seem like human universals; for instance, by my reading it’s an attitude entirely foreign to the perspective of the Bible, both the Hebrew Bible and the New Testament.