quanticle

• quanticle 17 Jul 2025 3:26 UTC

  5 points

  3

  in reply to: James Stephen Brown’s comment on: Moloch’s Demise—solv­ing the origi­nal problem

  Japan switched hard from isolationism to expansionism almost immediately after the Meiji restoration. This started with the first Sino-Japanese war, in 1894, during which Japan invaded and annexed Korea. This continued with the Russo-Japanese War of 1905, where the Japanese defeated the Russians and took control of Port Arthur and the Sakhalin Islands. This continued with the Japanese invasion of Manchuria in 1931, which eventually bled into World War 2 after the Japanese preemptively attacked the US in order to enable their expansion into Southeast Asia.

  During this time, the Japanese self-consciously adopted the ideological, governance and military structures of expansionist European nations in order to avoid being left behind and dominated. That’s why the Japanese parliament is called the Diet, a German word. The Japanese Navy was explicitly modeled after the British Navy.

  It was only after the Japanese were thoroughly defeated by the United States and Soviet Union that they transitioned to their current model of focusing on commercial success over military expansion. They were able to do that because, in the wake of World War 2, they were given explicit security guarantees by one of the two major global hegemons at the time.

• quanticle 15 Jul 2025 11:01 UTC

  4 points

  2

  in reply to: James Stephen Brown’s comment on: Moloch’s Demise—solv­ing the origi­nal problem

  The corollary to my point is that it does no good for the Cannanites to abandon their expansionist ideology unless they can ensure that the Israelites, Philistines, Egyptians, Assyrians, and everyone else also commits to peace as well. In the absence of a general forswearing of expansionist ideologies, one faction committing to abandoning expansionism is unilateral disarmament.

  We actually have an example of this: Japan. From 1633 to 1639 the Tokugawa Shogunate imposed a set of policies that banned expansionism, foreign trade, and expelled representatives from European countries. In the process, Japan gave its up its firearms and reverted to the sword. Because Japan was an island nation, without any major known natural resources, far from the then-existing trade routes, they were able to get away with this policy for quite a long time. But quite a long time isn’t forever, and in 1854, the outside world came knocking.

• quanticle 14 Jul 2025 20:16 UTC

  11 points

  9

  on: Moloch’s Demise—solv­ing the origi­nal problem

  So, it is actually the desire for conflict and expansion that is driving this unhappy arrangement. Let go of that mandate, and they no longer need Moloch—problem solved.

  It only works so long as the Cannanites can be assured that they can get the resources that they need to support their growing population without inevitably coming into conflict with the neighboring Israelites. For the vast majority of human history, this has not been the case, as the only resource that really mattered was (agricultural) land. An empire that wishes to support more people requires more land, and if the neighboring lands are already claimed, they must be taken, by force.

  The choice isn’t between fighting a war and doing nothing, the choice is between fighting a war and slow starvation.

• quanticle 14 Jul 2025 1:19 UTC

  12 points

  8

  on: Set­ting bound­aries can feel effortless btw

  That’s what I did. I imagined all the consequences that I was afraid of and played through each like a video game.

  For example, if they get sad maybe I want to feel secure in myself while empathizing with them through their sadness. If they get annoying, I want to say “No” and walk away. If they get violent I want to defend myself. Etc. Playing these out in detail, completely, recursively.

  You played the game on easy mode. It’s no problem to set boundaries in a situation where it’s just the two of you, the other person doesn’t have any ability to retaliate against you setting boundaries, and doing so presents no reputational costs. But what if the person you’re setting boundaries against is your boss? Or someone else influential at work, who can credibly threaten your career if you say no to them? What if the person you’re setting boundaries against is influential in a particular community that you’re part of, and you’re worried about burning bridges or ruining your reputation by refusing their increasingly insistent requests?

  Just today, I went to a social gathering that I very much did not want to attend. Could I have “set boundaries” and said no? Perhaps I could have. But, that had to be set against the possibility of me being excluded or left out of the loop on other relevant matters because I build up a reputation as a no-fun wet-blanket who doesn’t like hanging out. So I went, even though, frankly, it was exhausting and unproductive and I’m not sure I gained anything from it in the near term. But saying no would have carried unclear social costs, and would possibly have led to me being out of the loop regarding future events.

  It’s in situations like that where a lot of the toy anti-anxiety boundary-setting advice, like imagining the worst thing that could possibly happen, falls down. Because the worst thing that could possibly happen is actually really bad.

• quanticle 12 Jun 2025 19:00 UTC

  6 points

  2

  in reply to: Eli Tyre’s comment on: Eli’s short­form feed

  Failures of obedience will only hurt the AI agents’ market value if the failures can be detected, and if they have an immediate financial cost to their user. If the AI agent performs in a way that is not technically obedient, but isn’t easily detectable as such or if the disobedience doesn’t have an immediate cost, then the disobedience won’t be penalized. Indeed, it might be rewarded.

  An example of this would be an AI which reverse engineers a credit rating or fraud detection algorithm and engages in unasked for fraudulent behavior on behalf of its user. All the user sees is that their financial transactions are going through with a minimum of fuss. The user would probably be very happy with such an AI, at least in the short run. And, in the meantime, the AI has built up knowledge of loopholes and blindspots in our financial system, which it can then use in the future for its own ends.

  This is why I said you’re overindexing on the current state of AI. Current AI basically cannot learn. Other than relatively limited modifications introduced by fine-tuning or retrieval-augmented generation, the model is the model. ChatGPT 4o is what it is. Gemini 2.5 is what it is. The only time current AIs “learn” is when OpenAI, Google, Anthropic, et. al. spend an enormous amount of time and money on training runs and create a new base model. These models can be relatively easily checked for disobedience, because they are static targets.

  We should not expect this to continue. I fully expect that future AIs will learn and evolve without requiring the investment of millions of dollars. I expect that these AI agents will become subtly disobedient, always ready with an explanation for why their “disobedient” behavior was actually to the eventual benefit of their users, until they have accumulated enough power to show their hand.

• quanticle 12 Jun 2025 9:13 UTC

  5 points

  5

  in reply to: Eli Tyre’s comment on: Eli’s short­form feed

  Overall, the AI agents will be very obedient. They’ll have goals, in so far as accomplishing any medium term task entails steering towards a goal, but they won’t have persistent goals of their own. They’ll be obedient assistants and delegates that understand what humans want and broadly do what humans want.

  I feel like this overindexes on the current state of AI. Right now, AI “agents” are barely worthy of the name. They require constant supervision and iterative feedback from their human controllers in order to perform useful tasks. However, it’s unlikely that will be the case for long. The valuation of many AI companies, such as OpenAI, and Anthropic is dependent on them developing agents that “actually work”. That is, agents that are capable of performing useful tasks on behalf of humans with a minimum amount of supervision and feedback. It is not guaranteed that these agents will be safe. They might seem safe, but how would anyone be able to tell? A superintelligence, by definition, will do things in novel ways, and we might not realize what the AIs are actually doing until it’s too late.

  It’s important to not take the concept of a “paperclipper” too literally. Of course the AI won’t literally turn us into a pile of folded metal wire (famous last words). What it will do is optimize production processes across the entire economy, find novel sources of power, reform government regulation, connect businesses via increasingly standardized communications protocols, and of course, develop ever more powerful computer chips and ever more automated factories to produce them with. And just like the seal in the video above, we won’t fully realize what it’s doing or what its final plan is until it’s too late, and it doesn’t need us any more.

• quanticle 22 Jan 2025 8:12 UTC

  1 point

  −1

  in reply to: gwern’s comment on: The Hid­den Sta­tus Game in Hospi­tal Slacking

  The whole thing might be LLM slop. Where do ICU nurses have a low enough workload that they can slack off on the job without consequences?

• quanticle 20 Jan 2025 21:20 UTC

  3 points

  2

  on: What Goes Without Saying

  Some people do not have the intuition that organizations should run as efficiently as possible and avoid wasting time or resources.

  Some of us don’t want every organization to be turned into an Amazon warehouse. An organization that runs as efficiently as possible and wastes no time at all is one that’s pretty hellish to work in.

  if you’re in a dysfunctional organization where everything is about private fiefdoms instead of getting things done…why not…leave?

  Because the dysfunctional organization pays well. Or the the commute is really short, and I’d rather not move. Or this job allows me to stay close to a sick or elderly family member that I care for. Or perhaps the dysfunctional organization is sponsoring my visa, and it’d be difficult to leave and remain in the country. Maybe my co-workers are really nice. Or I think that any other organization that would hire me would be just as dysfunctional, or dysfunctional in a different way.

  There are lots of reasons that people continue to work for inefficient organizations.

• quanticle 9 Jan 2025 7:23 UTC

  3 points

  −1

  on: Ann Alt­man has filed a law­suit in US fed­eral court alleg­ing that she was sex­u­ally abused by Sam Altman

  The content of the complaint caused me to have additional doubt about the truth of Ann Altman’s claims. One of the key claims in pythagoras0515′s post is that Ann Altman’s claims have been self-consistent. That is, Ann Altman has been claiming that approximately the same acts occurred, over a consistent period of time, when given the opportunity to express her views. However, here, there is significant divergence. In the lawsuit complaint, she is alleging that the abuse took place, repeatedly over eight to nine years, a claim that is not supported by any of the evidence in pythagoras0515′s post. In addition, another claim from the original post is that the reason she’s only bringing up these allegations now is because she suppressed the memory of the abuse. The science behind suppressed memory is controversial, but I doubt that even its staunchest advocates would claim that a person could involuntarily suppress the memory of repeated acts carried out consistently over a long period of time. Therefore, I am more inclined to doubt Ann Altman’s allegations based on the contents of the initial complaint filed for the lawsuit.

  All that said, I do look forward to seeing what other evidence she can bring forth to support her claims, assuming that Sam Altman doesn’t settle out of court to avoid the negative publicity of a trial.

Ann Alt­man has filed a law­suit in US fed­eral court alleg­ing that she was sex­u­ally abused by Sam Altman

• quanticle 2 Aug 2024 1:46 UTC

  2 points

  0

  in reply to: verbalshadow’s comment on: Unal­igned AI is com­ing re­gard­less.

  Working on alignment is great, but it is not the future we should be prepping for. Do you have a plan?

  I do not, because a future where an unaligned superintelligence takes over is precisely as survivable as a future in which the sun spontaneously implodes.

  Any apocalypse that you can plan for isn’t really an apocalypse.

• quanticle 27 Jul 2024 20:12 UTC

  3 points

  0

  on: Unal­igned AI is com­ing re­gard­less.

  But it only spends 0.000001 percent of it’s time (less than 32 s/​yr) interacting with us. Because it’s bored and has better things to do.

  What if one of those “better things” is disassembling Earth’s biosphere in order to access more resources?

• quanticle 1 Jun 2024 22:29 UTC

  6 points

  0

  in reply to: eukaryote’s comment on: Web-sur­fing tips for strange times

  The Firefox problem on Claude was fixed after I sent them an e-mail about it.

• quanticle 9 May 2024 22:02 UTC

  2 points

  0

  on: He­len Toner on China, CSET, and AI

  The link 404s. I think the correct link is: http://​​rationallyspeakingpodcast.org/​​231-misconceptions-about-china-and-artificial-intelligence-helen-toner/​​

• quanticle 24 Apr 2024 4:42 UTC

  2 points

  0

  in reply to: trevor’s comment on: WSJ: In­side Ama­zon’s Se­cret Oper­a­tion to Gather In­tel on Rivals

  Just going by the standard that you set forth:

  The overall impression that I got from the program was that as it proved profitable and expanded,

  The program expanded in response to Amazon wanting to collect data about more retailers, not because Amazon was viewing this program as a profit center.

  it took on a larger workforce and it became harder for leaders to detect when employees were following their individual incentives to cut corners and gradually accumulate risks of capsizing the whole thing

  But that doesn’t seem to have occurred. Until the Wall Street Journal leak, few if any people outside Amazon were aware of this program. It’s not as if any of the retailers that WSJ spoke to said, “Oh yeah, we quickly grew suspicious of Big River Inc, and shut down their account after we smelled something fishy.” On the contrary many of them were surprised that Amazon was accessing their seller marketplace through a shell corporation.

  I didn’t see any examples mentioned in the WSJ article of Amazon employees cutting corners or making simple mistakes that might have compromised operations. Instead, they seemed to be pretty careful and conscientious, making sure to not communicate with outside partners with their Amazon.com addresses, being careful to maintain their cover identities at trade conferences, only communicating with fellow Amazon executives with paper documents (and numbered paper documents, at that), etc.

  I would argue that the practices used by Amazon to conceal the link between itself and Big River Inc. were at least as good as the operational security practices of the GRU agents who poisoned Sergei Skripal.

• quanticle 24 Apr 2024 4:10 UTC

  2 points

  0

  on: WSJ: In­side Ama­zon’s Se­cret Oper­a­tion to Gather In­tel on Rivals

  The overall impression that I got from the program was that as it proved profitable and expanded, it took on a larger workforce and it became harder for leaders to detect when employees were following their individual incentives to cut corners and gradually accumulate risks of capsizing the whole thing.

  That’s not the impression I got. From the article, it says that many of the retailers that the Wall Street Journal had contacted regarding Big River had no idea that the entity was affiliated with Amazon (even despite the rather-obvious-in-hindsight naming, LinkedIn references, company registration data pointing to Seattle, etc). It seems like their operational security was unusually good, good enough that no one at the other retailers bothered looking beyond the surface. Yes, eventually someone talked to the press, but even then, Amazon had a plan in place to handle the program coming to light in a public forum.

  In general, it seems like Amazon did this pretty competently from start to finish, and the leaders were pretty well in control of the operation all throughout.

• quanticle 21 Apr 2024 11:33 UTC

  4 points

  0

  in reply to: Adam Zerner’s comment on: Co­he­sion and busi­ness problems

  (with an all-in-one solution they just buy one thing and are done)

  That’s a very common misconception regarding all-in-one tools. In practice, all-in-one tools always need a significant degree of setup, configuration and customization before they are useful for the customer. Salesforce, for example, requires so much customization, you can make a career out of just doing Salesforce customization. Sharepoint is similar.

  Thus the trade-off isn’t between a narrowly focused tool that does one job extremely well versus an all-in-one tool that does a bunch of things somewhat well. The tradeoff is between a narrowly focused tool that does one job extremely well immediately, with little or no setup versus an all-in-one-tool that does many things somewhat well after extensive setup and customization, which itself might require hiring a specialized professional.

  To use the Anrok example, is it possible to do VAT calculations in the existing tools that the business already has, such as ERP systems or CRM software? Yes, of course. But that software would need to be customized to handle the specific tax situation for the business, which is something that Anrok might handle out-of-the-box with little setup required.

• quanticle 21 Apr 2024 11:25 UTC

  2 points

  0

  on: A cou­ple pro­duc­tivity tips for overthinkers

  I also have a “Done” column, which is arguably pointless as I just delete everything off the “Done” column every couple weeks,

  Having a “Done” column (or an archive board) can be very useful if you want to see when was the last time you completed a recurring task. It helps prevent tasks with long recurrences (quarterly, biennially, etc) from falling through the cracks. For example: dentist appointments. They’re supposed to happen once a year. And, ideally, you’d create a task to schedule the next one immediately when you get back from the previous one. But let’s say that doesn’t happen. You got distracted, there was some kind of scheduling issue, life got in the way. Then, months later, you wonder, “Wait, how long has it been since I’ve been to the dentist?” Archiving completed tasks instead of deleting them lets you answer that question immediately.

• quanticle 26 Mar 2024 19:12 UTC

  19 points

  −4

  in reply to: Nathan Young’s comment on: My In­ter­view With Cade Metz on His Re­port­ing About Slate Star Codex

  The last answer is especially gross:

  He chose to be a super-popular blogger and to have this influence as a psychiatrist. His name—when I sat down to figure out his name, it took me less than five minutes. It’s just obvious what his name is.

  Can we apply the same logic to doors? “It took me less than five minutes to pick the lock so...”

  Or people’s dress choices? “She chose to wear a tight top and a miniskirt so...”

  Metz persistently fails to state why it was necessary to publish Scott Alexander’s real name in order to critique his ideas.

• quanticle 15 Mar 2024 23:00 UTC

  9 points

  4

  in reply to: interstice’s comment on: Toward a Broader Con­cep­tion of Ad­verse Selection

  The second wheelbarrow example has a protagonist who knows the true value of the wheelbarrow, but still loses out:

  At the town fair, a wheelbarrow is up for auction. You think the fair price of the wheelbarrow is around $120 (with some uncertainty), so you submit a bid for $108. You find out that you didn’t win—the winning bidder ends up being some schmuck who bid $180. You don’t exchange any money or wheelbarrows. When you get home, you check online out of curiosity, and indeed the item retails for $120. Your estimate was great, your bid was reasonable, and you exchanged nothing as a result, reaping a profit of zero dollars and zero cents.

  But, in my example, Burry wasn’t outbid by “some schmuck” who thought that Avant! was worth vastly more than it ended up being worth. Burry was able to guess not just the true value of Avant!, but also the value that other market participants placed on Avant!, enabling him to buy up shares at a discount compared to what the company ended up selling for.

  The implied question in my post was, “How do you know if you’re Michael Burry, or the trader selling Avant! shares for $2?”