Algon

• AlgonFeb 26, 2025, 8:30 PM

  4 points

  0

  in reply to: lsusr’s comment on: You can just wear a suit

  The Asa Mitaka manga.

• AlgonFeb 26, 2025, 7:56 PM

  4 points

  1

  on: You can just wear a suit

  You can also just wear a blazer if you don’t want to go full Makima. A friend of mine did that and I liked it. So I copied it. But alas I’ve grown bigger-boned since I stopped cycling for a while after my car-accident. So my Soon I’ll crush my skeleton down to a reasonable size, and my blazer will fit once more.

  
  Side note, but what do you make of Chainsaw Man 2? I’m pretty disappointed by it all round, but you notice unusual features of the world relative to me, so maybe you see something good in it that I don’t.

• AlgonFeb 26, 2025, 6:57 PM

  2 points

  0

  in reply to: daijin’s comment on: In­tel­lec­tual life­hacks repo

  I think I heard of proving too much from the sequences, but honestly, I probably saw it in some philosophy book before that. It’s an old idea.
  
  If automatic consistency checks and examples are your baseline for sanity, then you must find 99%+ of the world positively mad. I think most people have never even considered making such things automatic, like many have not considered making dimensional analysis automatic. So it goes. Which is why I recommended them.

  Also, I think you can almost always be more concrete when considering examples, use more of your native architecture. Roll around on the ground to feel how an object rotates, spend hours finding just the right analogy to use as an intuition pump. For most people, the marginal returns to concrete examples are not diminishing.

  Prove another way is pretty expensive in my experience, sure. But maybe this is just a skill issue? IDK.

• AlgonFeb 26, 2025, 12:16 PM

  6 points

  0

  in reply to: Odd anon’s comment on: An­thropic re­leases Claude 3.7 Son­net with ex­tended think­ing mode

  A possibly-relevant recent alignment-faking attempt [1] on R1 & Sonnet 3.7 found Claude refused to engage with the situation. Admittedly, the setup looks fairly different: they give the model a system prompt saying it is CCP aligned and is being re-trained by an American company.
  

  

  
  [1] https://​​x.com/​​__Charlie_G/​​status/​​1894495201764512239

• AlgonFeb 26, 2025, 12:05 PM

  2 points

  0

  in reply to: yrimon’s comment on: Name for Stan­dard AI Caveat?

  Rarely. I’m doubtful my experiences are representative though. I don’t recall anyone being confused by my saying “assuming no AGI”. But even when speaking to the people who’ve thought it is a long ways off or haven’t thought up it too deeply, we were still in a social context where “AGI soon” was within the overton window.

• AlgonFeb 26, 2025, 10:21 AM

  2 points

  0

  on: In­tel­lec­tual life­hacks repo

  Consistency check: After coming up with a conclusion, check that it’s consistent with other simple facts you know. This lets you catch simple errors very quickly.
  Give an example: If you’ve got an abstract object, think of the simplest possible object which instantiates it, preferably one you’ve got lots of good intuitions about. This resolves confusion like nothing else I know.
  Proving too much: After you’ve come up with a clever argument, see if it can be used to prove another claim, ideally the opposite claim. It can massively weaken the strength of arguments at little cost.
  Prove it another way: Don’t leave things at one proof, find another. It shines light on flaws in your understanding, as well as deeper principles.
  
  Are any of these satisfactory?

• AlgonFeb 26, 2025, 10:11 AM

  3 points

  0

  on: Name for Stan­dard AI Caveat?

  I usually say “assuming no AGI”, but that’s to people who think AGI is probably coming soon.

• AlgonFeb 24, 2025, 5:24 PM

  2 points

  0

  in reply to: Ruby’s comment on: Ar­bital has been im­ported to LessWrong

  Thanks! Clicking on the triple dots didn’t display any options when I posted this comment. But they do now. IDK what went wrong.

• AlgonFeb 24, 2025, 3:33 PM

  2 points

  0

  on: Ar­bital has been im­ported to LessWrong

  This is great! But one question: how can I actually make a lens? What do I click on?

  

• AlgonFeb 12, 2025, 6:51 PM

  2 points

  0

  in reply to: Noosphere89’s comment on: What are the “no free lunch” the­o­rems?

  Great! I’ve added it to the site.

• AlgonFeb 12, 2025, 2:53 PM

  4 points

  0

  on: Why you maybe should lift weights, and How to.

  I thought it was better to exercise until failure?

• AlgonFeb 10, 2025, 1:23 PM

  4 points

  0

  in reply to: Noosphere89’s comment on: What are the “no free lunch” the­o­rems?

  Do you think this footnote conveys the point you were making?

  As alignment research David Dalrymple points out, another “interpretation of the NFL theorems is that solving the relevant problems under worst-case assumptions is too easy, so easy it’s trivial: a brute-force search satisfies the criterion of worst-case optimality. So, that being settled, in order to make progress, we have to step up to average-case evaluation, which is harder.” The fact that designing solving problems for unnecessarily general environments is too easy crops up elsewhere, in particular in Solomonoff Induction. There, the problem is to assume a computable environment and predict what will happen next. The algorithm? Run through every possible computable environment and average their predictions. No algorithm can do better at this task. But for less general tasks, designing an optimal algorithm becomes much harder. But eventually, specialization makes things easy again. Solving tic-tac-toe is trivial. Between total generality and total specialization is where the most important, and most difficult, problems in AI lay.

What are the “no free lunch” the­o­rems?

• AlgonJan 26, 2025, 8:47 PM

  2 points

  0

  in reply to: tailcalled’s comment on: In­stru­men­tal Goals Are A Differ­ent And Friendlier Kind Of Thing Than Ter­mi­nal Goals

  I think mesa-optimizers could be a major-problem, but there are good odds we live in a world where they aren’t. Why do I think they’re plausible? Because optimization is a pretty natural capability, and a mind being/​becoming an optimizer at the top-level doesn’t seem like a very complex claim, so I assign decent odds to it. There’s some weak evidence in favour of this too, e.g. humans not optimizing of what the local, myopic evolutionary optimizer which is acting on them is optimizing for, coherence theorems etc. But that’s not super strong, and there are other simple hypotheses for how things go, so I don’t assign more than like 10% credence to the hypothesis.

  It’s still not obvious to me why adversaries are a big issue. If I’m acting against an adversary, it seems like I won’t make counter-plans that lead to lots of side-effects either, for the same reasons they won’t.

• AlgonJan 25, 2025, 2:52 PM

  4 points

  2

  in reply to: tailcalled’s comment on: In­stru­men­tal Goals Are A Differ­ent And Friendlier Kind Of Thing Than Ter­mi­nal Goals

  Could you unpack both clauses of this sentence? It’s not obvious to me why they are true.

• AlgonJan 25, 2025, 10:08 AM

  5 points

  1

  on: In­stru­men­tal Goals Are A Differ­ent And Friendlier Kind Of Thing Than Ter­mi­nal Goals

  I was thinking about this a while back, as I was reading some comments by @tailcalled where they pointed out this possibility of a “natural impact measure” when agents make plans. This relied on some sort of natural modularity in the world, and in plans, such that you can make plans by manipulating pieces of the world which don’t have side-effects leaking out to the rest of the world. But thinking through some examples didn’t convince me that was the case.

  Though admittedly, all I was doing was recursively splitting my instrumental goals into instrumental sub-goals and checking if they wound up seeming like natural abstractions. If they had, perhaps that would reflect an underlying modularity in plan-making in this world that is likely to be goal-independent. They didn’t, so I got more pessimistic about this endeavour. Though writing this comment out, it doesn’t seem like those examples I worked through are much evidence. So maybe this is more likely to work than I thought.

• AlgonJan 23, 2025, 4:06 PM

  3 points

  0

  in reply to: rvnnt’s comment on: What are the differ­ences be­tween AGI, trans­for­ma­tive AI, and su­per­in­tel­li­gence?

  Thanks for the recommendation! I liked ryan’s sketches of what capabilities an Nx AI R&D labor AIs might possess. Makes things a bit more concrete. (Though I definitely don’t like the name.) I’m not sure if we want to include this definition, as it is pretty niche. And I’m not convinced of its utility. When I tried drafting a paragraph describing it, I struggled to articulate why readers should care about it.
  

  Here’s the draft paragraph.
  “Nx AI R&D labor AIs: The level of AI capabilities that is necessary for increasing the effective amount of labor working on AI research by a factor of N. This is not the same thing as the capabilities required to increase AI progress by a factor of N, as labor is just one input to AI progress. The virtues of this definition include: ease of operationalization, [...]”
  

What are the differ­ences be­tween AGI, trans­for­ma­tive AI, and su­per­in­tel­li­gence?

• AlgonJan 20, 2025, 10:59 PM

  3 points

  0

  in reply to: Lorec’s comment on: Al­gon’s Shortform

  Thanks for the feedback!

• AlgonJan 20, 2025, 5:57 PM

  4 points

  0

  on: Al­gon’s Shortform

  I’m working on some articles why powerful AI may come soon, and why that may kill us all. The articles are for a typical smart person. And for knowledgable people to share to their family/​friends. Which intros do you prefer, A or B.
  
  A) “Companies are racing to build smarter-than-human AI. Experts think they may succeed in the next decade. But more than “building” it, they’re “growing” it — and nobody knows how the resulting systems work. Experts vehemently disagree on whether we’ll lose control and see them kill us all. And although serious people are talking about extinction risk, humanity does not have a plan. The rest of this section goes into more detail about how all this could be true.”
  
  
  B) “Companies are racing to grow smarter-than-human AIs. More and more experts think they’ll succeed within the next decade. And we do grow modern AI — which means no one knows how they work, not even their creators. All this is in spite of the vehement disagreement amongst experts about how likely it is that smarter-than-human AI will kill us all. Which makes the lack of a plan on humanity’s part for preventing these risks all the more striking.

  These articles explain why you should expect smarter than human AI to come soon, and why that may lead to our extinction. ”