lc

Karma: 10,135

lc Apr 9, 2025, 8:49 PM
4 points
4
in reply to: johnswentworth’s comment on: Navigation by Moonlight

Now, one could reasonably counter-argue that the yin strategy delivers value somewhere else, besides just e.g. “probability of a date”. Maybe it’s a useful filter for some sort of guy...

I feel like you know this is the case and I’m wondering why you’re even asking the question. Of course it’s a filter; the entire mating process is. Women like confidence, and taking these mixed signals as a sign of attraction is itself a sign of confidence. Walking up to a guy and asking him to have sex immediately would also be more “efficient” by some deranged standards, but the point of flirting is that you get to signal social grace, selectivity, and a willingness to walk away from the interaction.

lc Apr 9, 2025, 4:09 PM
14 points
2
in reply to: leogao’s comment on: leogao’s Shortform
Close friend of mine, a regular software engineer, recently threw tens of thousands of dollars—a sizable chunk of his yearly salary—at futures contracts on some absurd theory about the Japanese Yen. Over the last few weeks, he coinflipped his money into half a million dollars. Everyone who knows him was begging him to pull out and use the money to buy a house or something. But of course yesterday he sold his futures contracts and bought into 0DTE Nasdaq options on another theory, and literally lost everything he put in and then some. I’m not sure but I think he’s down about half his yearly salary overall.

He has been doing this kind of thing for the last two years or so—not just making investments, but making the most absurd, high risk investments you can think of. Every time he comes up with a new trade, he has a story for me about how his cousin/whatever who’s a commodities trader recommended the trade to him, or about how a geopolitical event is gonna spike the stock of Lockheed Martin, or something. On many occasions I have attempted to explain some kind of Inadequate Equilibria thesis to him, but it just doesn’t seem to “stick”.

It’s not that he “rejects” the EMH in these conversations. I think for a lot of people there is literally no slot in their mind that is able to hold market efficiency/inefficiency arguments. They just see stocks moving up and down. Sometimes the stocks move in response to legible events. They think, this is a tractable problem, I just have to predict the legible events. How could I be unable to make money? Those guys from The Big Short did!

He is also taking a large amount of stimulants. I think that is compounding the situation a bit.

lc Apr 8, 2025, 3:41 PM
16 points
11
in reply to: habryka’s comment on: AI 2027: What Superintelligence Looks Like
Typically I operationalize “employable as a software engineer” as being capable of completing tasks like:
- “Fix this error we’re getting on BetterStack.”
- “Move our Redis cache from DigitalOcean to AWS.”
- “Add and implement a cancellation feature for ZeroPath scans.”
- “Add the results of this evaluation to our internal benchmark.”
These are pretty representative examples of the kinds of tasks your median software engineer will be getting and resolving on a day to day basis.

No chatbot or chatbot wrapper can complete tasks like these for an engineering team at present, incl. Devin et. al. Partly this is because most software engineering work is very high-context, in the sense that implementing the proper solution depends on understanding a large body of existing infrastructure, business knowledge, and code.

When people talk about models today doing “agentic development”, they’re usually explaining its ability to complete small projects in low-context situations, where all you need to understand is the prompt itself and software engineering as a discipline. That makes sense, because if you ask AIs to write (for example) a PONG game in javascript, the AI can complete each of the pieces in one pass, and fit everything it’s doing into one context window. But that kind of task is unlike the vast majority of things employed software engineers do today, which is why we’re not experiencing an intelligence explosion right this second.
What links here?
- lc's comment on Reactions to METR task length paper are insane by Cole Wyeth (Apr 11, 2025, 11:59 PM; 18 points)

lc Apr 5, 2025, 5:19 AM
5 points
0
on: AI 2027: What Superintelligence Looks Like

They strengthen chip export restrictions, order OpenBrain to further restrict its internet connections, and use extreme measures to secure algorithmic progress, like wiretapping OpenBrain employees—this catches the last remaining Chinese spy

Wiretapping? That’s it? Was this spy calling Xi from his home phone? xD

lc Apr 1, 2025, 9:40 PM
119 points
36
in reply to: habryka’s comment on: LessWrong has been acquired by EA
As a newly minted +100 strong upvote, I think the current karma economy accurately reflects how my opinion should be weighted

lc Apr 1, 2025, 7:23 PM
4 points
0
in reply to: habryka’s comment on: lc’s Shortform
I have Become Stronger

lc Apr 1, 2025, 6:37 PM
47 points
0
on: lc’s Shortform
My strong upvotes are now giving +1 and my regular upvotes give +2.

lc Apr 1, 2025, 6:15 PM
7 points
0
in reply to: Rafael Harth’s comment on: Rafael Harth’s Shortform
test

lc Mar 26, 2025, 3:50 PM
11 points
2
in reply to: ryan_greenblatt’s comment on: Recent AI model progress feels mostly like bullshit
Just edited the post because I think the way it was phrased kind of exaggerated the difficulties we’ve been having applying the newer models. 3.7 was better, as I mentioned to Daniel, just underwhelming and not as big a leap as either 3.6 or certainly 3.5.

lc Mar 25, 2025, 12:36 AM
12 points
0
in reply to: Daniel Kokotajlo’s comment on: Recent AI model progress feels mostly like bullshit
If you plot a line, does it plateau or does it get to professional human level (i.e. reliably doing all the things you are trying to get it to do as well as a professional human would)?
It plateaus before professional human level, both in a macro sense (comparing what ZeroPath can do vs. human pentesters) and in a micro sense (comparing the individual tasks ZeroPath does when it’s analyzing code). At least, the errors the models make are not ones I would expect a professional to make; I haven’t actually hired a bunch of pentesters and asked them to do the same tasks we expect of the language models and made the diff. One thing our tool has over people is breadth, but that’s because we can parallelize inspection of different pieces and not because the models are doing tasks better than humans.
What about 4.5? Is it as good as 3.7 Sonnet but you don’t use it for cost reasons? Or is it actually worse?
We have not yet tried 4.5 as it’s so expensive that we would not be able to deploy it, even for limited sections.

lc Mar 25, 2025, 12:00 AM
10 points
0
in reply to: Daniel Kokotajlo’s comment on: Recent AI model progress feels mostly like bullshit
We use different models for different tasks for cost reasons. The primary workhorse model today is 3.7 sonnet, whose improvement over 3.6 sonnet was smaller than 3.6′s improvement over 3.5 sonnet. When taking the job of this workhorse model, o3-mini and the rest of the recent o-series models were strictly worse than 3.6.

Recent AI model progress feels mostly like bullshit

lcMar 24, 2025, 7:28 PM

313 points

80 comments8 min readLW link

(zeropath.com)

lc Mar 23, 2025, 7:14 PM
7 points
1
in reply to: brambleboy’s comment on: lc’s Shortform
I haven’t read the METR paper in full, but from the examples given I’m worried the tests might be biased in favor of an agent with no capacity for long term memory, or at least not hitting the thresholds where context limitations become a problem:
For instance, task #3 here is at the limit of current AI capabilities (takes an hour). But it’s also something that could plausibly be done with very little context; if the AI just puts all of the example files in its context window it might be able to write the rest of the decoder from scratch. It might not even need to have the example files in memory while it’s debugging its project against the test cases.
Whereas a task to fix a bug in a large software project, while it might take an engineer associated with that project “an hour” to finish, requires stretching the limits of the amount of information it can fit inside a context window, or recall beyond what we seem to be capable of doing today.
What links here?
- LLM AGI will have memory, and memory changes alignment by Seth Herd (Apr 4, 2025, 2:59 PM; 70 points)
- Noosphere89's comment on Recent AI model progress feels mostly like bullshit by lc (Mar 24, 2025, 10:22 PM; 14 points)

lc Mar 23, 2025, 5:15 PM
5 points
−5
on: lc’s Shortform
There was a type of guy circa 2021 that basically said that gpt-3 etc. was cool, but we should be cautious about assuming everything was going to change, because the context limitation was a key bottleneck that might never be overcome. That guy’s take was briefly “discredited” in subsequent years when LLM companies increased context lengths to 100k, 200k tokens.

I think that was premature. The context limitations (in particular the lack of an equivalent to human long term memory) are the key deficit of current LLMs and we haven’t really seen much improvement at all.

lc Mar 18, 2025, 4:52 AM
10 points
9
in reply to: DAL’s comment on: DAL’s Shortform

If AI executives really are as bullish as they say they are on progress, then why are they willing to raise money anywhere in the ballpark of current valuations?

The story is that they need the capital to build the models that they think will do that.

lc Feb 15, 2025, 1:22 PM
4 points
1
on: lc’s Shortform
Moral intuitions are odd. The current government’s gutting of the AI safety summit is upsetting, but somehow less upsetting to my hindbrain than its order to drop the corruption charges against a mayor. I guess the AI safety thing is worse in practice but less shocking in terms of abstract conduct violations.

lc Feb 14, 2025, 10:33 PM
4 points
0
in reply to: Viliam’s comment on: Virtue signaling, and the “humans-are-wonderful” bias, as a trust exercise
It helps, but this could be solved with increased affection for your children specifically, so I don’t think it’s the actual motivation for the trait.

The core is probably several things, but note that this bias is also part of a larger package of traits that makes someone less disagreeable. I’m guessing that the same selection effects that made men more disagreeable than women are also probably partly responsible for this gender difference.

lc Feb 14, 2025, 9:25 PM
4 points
0
in reply to: Viliam’s comment on: Virtue signaling, and the “humans-are-wonderful” bias, as a trust exercise

I suspect that the psychopath’s theory of mind is not “other people are generally nicer than me”, but “other people are generally stupid, or too weak to risk fighting with me”.

That is true, and it is indeed a bias, but it doesn’t change the fact that their assessment of whether others are going to hurt them seems basically well calibrated. The anecdata that needs to be explained is why nice people do not seem to be able to tell when others are going to take advantage of them, but mean people do. The posts’ offered reason is that generous impressions of others are advantageous for trust-building.

Mr. Portman probably believed that some children forgot to pay for the chocolate bars, because he was aware that different people have different memory skills.

This was the explanation he offered, yeah.

lc Feb 13, 2025, 9:43 AM
5 points
0
in reply to: sapphire’s comment on: Virtue signaling, and the “humans-are-wonderful” bias, as a trust exercise
This post is about a suspected cognitive bias and why I think it came to be. It’s not trying to justify any behavior, as far as I can tell, unless you think the sentiment “people are pretty awful” justifies bad behavior in of itself.

The game theory is mostly an extended metaphor rather than a serious model. Humans are complicated.

Virtue signaling, and the “humans-are-wonderful” bias, as a trust exercise

lcFeb 13, 2025, 6:59 AM

44 points

16 comments4 min readLW link

lc

Re­cent AI model progress feels mostly like bullshit

Virtue sig­nal­ing, and the “hu­mans-are-won­der­ful” bias, as a trust exercise

Recent AI model progress feels mostly like bullshit

Virtue signaling, and the “humans-are-wonderful” bias, as a trust exercise