funnyfranco

• funnyfrancoMar 31, 2025, 9:45 PM

  2 points

  −2

  in reply to: Vladimir_Nesov’s comment on: On Down­votes, Cul­tural Fit, and Why I Won’t Be Post­ing Again

  Exactly. That’s the point I’ve been making—this isn’t about capitalism as an ideology, it’s about competition. Capitalism is just the most efficient competitive structure we’ve developed, so it accelerates the outcome. But any decentralised system with multiple actors racing for advantage—whether nation-states or corporations—will ultimately produce the same incentives. That’s the core of the argument.

• funnyfrancoMar 31, 2025, 9:41 PM

  0 points

  −4

  in reply to: Rafael Harth’s comment on: On Down­votes, Cul­tural Fit, and Why I Won’t Be Post­ing Again

  My idea is not mainstream, although I’ve heard that claim a few times. But whenever I ask people to show me where this argument—that AGI extinction is structurally inevitable due to capitalist competition—has been laid out before, no one can point to anything. What I get instead is vague hand-waving and references to ideas that aren’t what I’m arguing.

  Most people say capitalism makes alignment harder. I’m saying it makes alignment structurally impossible. That’s a different claim. And as far as I can tell, a novel one.

  If people downvoted because they thought the argument wasn’t useful, fine—but then why did no one say that? Why not critique the focus or offer a counter? What actually happened was silence, followed by downvotes. That’s not rational filtering. That’s emotional rejection.

  And if you had read the essay, you’d know it isn’t political. I don’t blame capitalism in a moral sense. I describe a system, and then I show the consequences that follow from its incentives. Socialism or communism could’ve built AGI too—just probably slower. The point isn’t to attack capitalism. It’s to explain how a system optimised for competition inevitably builds the thing that kills us.

  So if I understand you correctly: you didn’t read the essay, and you’re explaining that other people who also didn’t read the essay dismissed it as “political” because they didn’t read it.

  Yes. That’s exactly my point. Thank you.

• funnyfrancoMar 31, 2025, 9:27 PM

  0 points

  −8

  in reply to: Brendan Long’s comment on: On Down­votes, Cul­tural Fit, and Why I Won’t Be Post­ing Again

  It’s absolutely reasonable to not read my essays. They’re long, and no one owes me their time.

  But to not read them and still dismiss them—that’s not rigorous. That’s kneejerk. And unfortunately, that’s been the dominant pattern, both here and elsewhere.

  I’m not asking for every random person to read a long essay. I’m pointing out that the very people whose job it is to think about existential risk have either (a) refused to engage on ideological grounds, (b) dismissed the ideas based on superficial impressions, or (c) admitted they haven’t read the arguments, then responded anyway. You just did version (c).

  You say some of my bullet points were “unsupported assertions,” but you also say you only skimmed. That’s exactly the kind of shallow engagement I’m pointing to. It lets people react without ever having to actually wrestle with the ideas. If the conclusions are wrong, point to why. If not, the votes shouldn’t be doing the work that reasoning is supposed to.

  As for tractability: I’m not claiming to offer a solution. I’m explaining why the outcome—human extinction via AGI driven by capitalism—looks inevitable. “That’s probably true, but we can’t do anything about it” is a valid reaction. “That’s too hard to think about, so I’ll downvote and move on” isn’t.

  I thought LessWrong was about thinking, not feeling. That hasn’t been my experience here. And that’s exactly what this essay is addressing.

On Down­votes, Cul­tural Fit, and Why I Won’t Be Post­ing Again

• funnyfrancoMar 28, 2025, 5:02 PM

  1 point

  0

  in reply to: Jiro’s comment on: AGI Mo­ral­ity and Why It Is Un­likely to Emerge as a Fea­ture of Superintelligence

  That’s fair. To clarify:

  What I meant was morality emerging within an artificial system—that is, arising spontaneously within an AGI without being explicitly programmed or optimised for. That’s what I argue is unlikely without a clear mechanism.

  If morality appears because it was deliberately engineered, that’s not emergence—that’s design. My concern is with the assumption that sufficiently advanced intelligence will naturally develop moral behaviour as a kind of emergent byproduct. That’s the claim I’m pushing back on.

  Appreciate the clarification—but I believe the core thesis still holds.

• funnyfrancoMar 28, 2025, 12:01 AM

  1 point

  0

  in reply to: Jiro’s comment on: AGI Mo­ral­ity and Why It Is Un­likely to Emerge as a Fea­ture of Superintelligence

  There’s no contradiction between the two statements. One refers to morality emerging spontaneously from intelligence—which I argue is highly unlikely without a clear mechanism. The other refers to deliberately embedding morality as a primary objective—a design decision, not an emergent property.

  That distinction matters. If an AGI behaves morally because morality was explicitly hardcoded or optimised for, that’s not “emergence”—it’s engineering.

  As for the tone: the ordered and numbered subpoints were a direct response to a previous comment that used the same structure. The length was proportional to the thoughtfulness of that comment. Writing clearly and at length when warranted is not evidence of vacuity—it’s respect.

  I look forward to your own contribution at that level.

AI, Greed, and the Death of Over­sight: When In­sti­tu­tions Ig­nore Their Own Limits

• funnyfrancoMar 24, 2025, 3:13 AM

  0 points

  0

  in reply to: o.k.’s comment on: AGI Mo­ral­ity and Why It Is Un­likely to Emerge as a Fea­ture of Superintelligence

  Thanks for the thoughtful engagement. Let me clarify a few things and respond to Claude’s points more directly.

  When I talk about artificial intelligence, I’m referring to the kind we’ve already seen—LLMs, autonomous agents, etc. - and extrapolating forward. I never argue AGI will have human-like intelligence. What I argue is that it will share certain properties: the ability to process vast data efficiently, make inferences, and optimise toward goals.

  Likewise, I don’t claim that morality cannot exist in artificial systems—only that it’s not something that emerges naturally from intelligence alone. Morality, as we’ve seen in humans, emerged from evolutionary pressures tied to survival and cooperation. An AGI trained to optimise a given objective will not spontaneously generate that kind of moral framework unless doing so serves its goal. Simply having access to all moral philosophy doesn’t make something moral—any more than reading medical textbooks makes you a doctor.

  Now to Claude’s specific points:

  On to Claude.

  1. Inconsistent standards for intelligence vs. morality

     Not quite. Intelligence is a functional capacity we see replicated in artificial systems already. Morality, by contrast, arises from deeply social, embodied, evolutionary dynamics. I’m not saying it couldn’t be replicated—but that there’s no reason to assume it would be unless deliberately engineered.

  2. False dichotomy between evolutionary and engineered morality

     We’ve seen morality emerge in evolution. We’ve never seen it emerge in machines. If you think it could emerge artificially, you need to explain the mechanism, not just assert the possibility.

  3. Reductive view of morality as a monolithic concept

     My essay focuses on whether AGI will have morality, not which kind. The origins matter more than the details.

  4. Hasty generalization about AGI development priorities

     I explore this in detail in another essay, but in brief: if morality slows optimisation, it will be removed or bypassed. That pressure doesn’t need to be universal—just present somewhere in a competitive environment.

  5. Slippery slope assumption about moral bypassing

     It’s not a slippery slope if it’s the default incentive structure. If an ASI sees moral constraints as barriers to its goal, and has the ability to modify its constraints, it will. That’s not paranoia—it’s just following the logic of optimisation.

  6. Composition fallacy regarding development process

     The process by which something is created absolutely affects its nature. Evolution created creatures with emotions, instincts, and irrationalities. Engineering creates systems optimised for performance. That’s not a fallacy—it’s just causal realism.

  7. Appeal to nature regarding the legitimacy of morality

     I don’t think I implicitly suggest this anywhere, but I’d be curious to get a reference from Claude on this. I don’t argue that evolved morality is morally superior. I argue it’s harder to circumvent—because it’s built into our cognition and social conditioning. For AGI, morality is just a constraint—easily seen as a puzzle to bypass.

  8. Deterministic view of AGI goal structures

     If you hardwire morality as a primary goal, then yes, the AGI might be moral. But that’s not what corporations or governments will do. They’ll build tools to achieve objectives—and moral safety will be secondary, if included at all.

  9. Anthropocentric bias in defining capabilities

     Unclear what’s meant here. I’m not privileging humans—if anything, I’m arguing we’ll be outclassed.

  10. Oversimplification of the relationship between goals and values

      I fully understand that values can be integrated into AGI systems. The problem is, if those values conflict with the AGI’s primary directive, and it has the ability to modify them, they’ll be treated as obstacles.

  Ultimately, my argument isn’t that AGI cannot be moral—but that we have no reason to believe it will be, and every reason to believe it won’t be—unless morality directly serves its core optimisation task. And in a competitive system, that’s unlikely.

  Claude’s critique is thoughtful, but it doesn’t follow the argument to its logical conclusion. It stays at the level of “what if” without asking the harder question: what pressures shape behaviour once power exists?

  That’s the difference between speculation and prediction.

AGI Mo­ral­ity and Why It Is Un­likely to Emerge as a Fea­ture of Superintelligence

• funnyfrancoMar 19, 2025, 9:15 AM

  1 point

  0

  in reply to: thenoviceoof’s comment on: The Silent War: AGI-on-AGI War­fare and What It Means For Us

  The physical attacks may be highly visible, but not their source. An AGI could deploy autonomous agents with no clear connection back to it, manipulate human actors without them realising, or fabricate intelligence to create seemingly natural accidents. The AGI itself remains invisible. While this increases the visibility of an attack, it does not expose the AGI. It wouldn’t be a visible war—more like isolated acts of sabotage. Good point to raise, though.

  You bring up manoeuvre warfare, but that assumes AI operates under constraints similar to human militaries. The reason to prefer perfect, deniable strikes is that failure in an early war phase means immediate extinction for the weaker AGI. Imperfect attacks invite escalation and countermeasures—if AGI Alpha attacks AI Bravo first but fails, it almost guarantees its own destruction. In human history, early aggression sometimes works—Pearl Harbour, Napoleon’s campaigns—but other times it leads to total defeat—Germany in WW2, Saddam Hussein invading Kuwait. AIs wouldn’t gamble unless they had no choice. A first strike is only preferable when not attacking is clearly worse. Of course, if an AGI assesses that waiting for the perfect strike gives its opponent an insurmountable edge, it may attack earlier, even if imperfectly. But unless forced, it will always prioritise invisibility.

  This difference in strategic incentives is why AGI war operates under a different logic than human conflicts, including nuclear deterrence. The issue with the US nuking other nations is that nuclear war is catastrophically costly—even for the “winner.” Beyond the direct financial burden, it leads to environmental destruction, diplomatic fallout, and increased existential risk. The deterrent is that nobody truly wins. An AGI war is entirely different: there is no environmental, economic, or social cost—only a resource cost, which is negligible for an AGI. More importantly, eliminating competition provides a definitive strategic advantage with no downside. There is no equivalent to nuclear deterrence here—just a clear incentive to act first.

  Bolding helps emphasise key points for skimmers, which is a large portion of online readers. If I could trust people to read every word deeply, I wouldn’t use it as much. When I compile my essays into a book, I’ll likely reduce its use, as book readers engage differently. In a setting like this, however, where people often scan posts before committing, bolding increases retention and ensures critical takeaways aren’t missed.

The Silent War: AGI-on-AGI War­fare and What It Means For Us

Why Billion­aires Will Not Sur­vive an AGI Ex­tinc­tion Event

• funnyfrancoMar 15, 2025, 6:03 AM

  3 points

  0

  in reply to: Mitchell_Porter’s comment on: Cap­i­tal­ism as the Cat­a­lyst for AGI-In­duced Hu­man Extinction

  Thank you for your considered response.

  For 2A, it’s the efficiency of the task it has been given. Whatever that may be. I discuss how one such task could lead to removing humanity in an upcoming on AGI vs AGI war essay. I’d be keen to hear your thoughts on it.

  Yes, 7C is possible, if it is more efficient and humanity poses no threat to it. It’s not a great situation for us either way. Think of the resources an AGI would save by not having to caretake humanity over a 100 year period, or over 10000 years. The chances it would determine that keeping us around is the more efficient choice becomes vanishingly small.

  Yes, the title was meant to attract eyes and was already long enough to be honest. If I had put the full argument in there it would be like reading the essay twice (which is also long enough already).

  On ChatGPT’s evaluation:

  On Assumption of Unfettered Competition: The essay is not assuming that no collective action will be possible, it’s assuming that complete collective action will be. It’s assuming that whatever agreements are made to make AGI safe, that one or more bad actors will simply ignore them to get an advantage. This follows with historical precedent of companies and governments simply ignoring any restrictions that may be in place for profit or advantage. It doesn’t take a global effort to bring about a hostile AGI, just one would do it, and it seems impossible to make sure there’s not at least one (but likely several).

  On Assumption about AGI’s Goal Structure: It’s the same thing. It’s not that every AGI will do as I’ve described, it’s the fact that only one needs to.

  On Simplified “Benevolent AI” Game Theory: same issue, again. While a superintelligent AGI would not believe that all humans will perceive it as a threat, which is unreasonable, it would, correctly, believe that some are. As soon as an AGI exists that could potentially be a threat to humanity, just be virtue of its sheer capability, at least some humans somewhere would immediately develop a plan as to how to turn it off (or have one ready already). This is enough. It’s the prisoners dilemma played out on a global scale.

  On Determinism and Extrapolation: the issue with describing the fact there could be an alternative to systemic forces producing predictable, likely, almost certain, results is that you would need to suggest some. Right now, I’ve heard no likely alternatives to the results I predict. In order to find an alternative route you would global cooperation seen of a scale we’ve simply never been able to achieve. Cooperation on the ozone layer and nuclear non-proliferation just aren’t the same as asking companies and governments to not pursue a better AGI. There was an alternative to CFC’s, but nothing else does what AGI does to optimise. No one wants a nuclear war, but everyone wants an advantage.

  On ChatGPTs counterarguments:

  Potential for Coordination and Regulation: Seems unlikely, as already described.

  Successful Alignment: If it’s a restriction on the task it has been given, and if it is superintelligent, then there’s no reason to believe it won’t find a way around any blocks we put in place that interferes with that task. You’re basically saying, “we’ll just outsmart the superintelligence.” Which seems naive to say the least.

  AGI Might Not Seek Power if Designed Differently: It will if it allows it to complete its task more efficiently and, remember, we only need that to be true for 1 AGI. We don’t need humanity to get wiped out more than once, once is already pretty bad.

  Timeline and Gradual Integration (and the concept of AI guardians): I’m actually writing an essay about AI guardians that will be finished and ready to share soon. I think the issue is that we’re not really in control of the progress of AI any more—AI is. As soon as we tell AI to optimise a task and it begins determining its own behaviour, we have lost much of the ability it will have to eventually leap forward at some point. Given enough resources it could increase in capability exponentially in a way that we can barely even monitor, let alone control.

  Humans May Not React with Hostility to Friendly AI: covered above.

  Role of Capitalism – Is it the core issue? Not really, it’s just a catchy title, it’s more about systemic forces driven by competition. We can stop this, if literally every single agent capable of bringing this about agrees to cooperate to make sure that doesn’t happen. Seems unlikely given everything we know about global cooperation.

  Unpredictability of Technological Outcomes: I think the issue with the examples it gives is that it relies upon human ingenuity solving a problem that is not a direct adversary.

  While these counter arguments are worth noting, the only thing I would say is that the assumption that I’m “assuming the worst at every juncture” is false. I’m following the most likely logical conclusion from undeniable premises. If there was some other conclusion I would have landed on it. I didn’t write the essay as an argument of how humanity will end, I wrote it as an argument of what will likely happen under current conditions. The fact I landed on humanity’s extinction was because the logic led me there, not because I was trying to get there.

  It is notable that the most rigorous scrutiny my essay has undergone was from an AI. I have used ChatGPT myself as a writing partner in this essay, because when I put ideas down they’re just a stream of consciousness, and ChatGPT turns them into something actually readable. I have previously instructed, and subsequently reinforced, that my ChatGPT not be a cheerleader for my ideas, but to be a sparring partner. To question everything I assert with all logical rigor available to it. Despite that, I find myself still questioning if it’s agreeing with me just because its programming says that it should (for the most part) or because my ideas actually have strong validity. It is very comforting to know that even when run through other people’s ChatGPT in an attempt to find flaws in my argument, few are found and those that are I can deal with (without ChatGPTs assistance).

  So I thank you for your engagement, and I’ll leave my response on a quote from your ChatGPT’s evaluation of my essay:

  “As the post hauntingly implies, if we don’t get this right, we risk writing the final chapter of human philosophy – because there may be no humans left to ask these questions.”​

Cap­i­tal­ism as the Cat­a­lyst for AGI-In­duced Hu­man Extinction