DirectedEvolution

Karma: 12,387

Pandemic Prediction Checklist: H5N1

Pandemic Prediction Checklist: Monkeypox

Correlation may imply some sort of causal link.

For guessing its direction, simple models help you think.

Controlled experiments, if they are well beyond the brink

Of .05 significance will make your unknowns shrink.

Replications show there’s something new under the sun.

Did one cause the other? Did the other cause the one?

Are they both controlled by what has already begun?

Or was it their coincidence that caused it to be done?

DirectedEvolution 1 Apr 2026 4:32 UTC
2 points
0
on: Is Bayesianism Susceptible to the Mail-Order Prophet Scam?
The likelihood function is a modeling choice. You are free to choose one that assigns a lower P(Not Scam|N Correct Predictions) than would be warranted for an unbiased model.
For this scam, even if they mailed letters to everyone on the planet, you could calculate the maximum number of guaranteed correct predictions they could have achieved if they are running this scam and were scamming a given fraction of the population. Then you could, for example, construct a model that assigns arbitrarily low probability to that number of letters and treats the later predictions as meaningful evidence.

DirectedEvolution 1 Apr 2026 3:24 UTC
5 points
0
in reply to: Mateusz Bagiński’s comment on: Mateusz Bagiński’s Shortform
I find myself paying far more attention to the LLM replies to questions I ask than I do to other text.
I think this is for a couple reasons.
1. The questions I’m asking are extremely specific to my niche interests at that exact moment, and I’m dead certain the LLM replies I‘m reading are the best available answers to my specific question that have ever been produced.
2. The LLM is generally sycophantic, but that registers to me as a sort of egoless confirmation bias that comes with easy flags to notice and interrogate. Human authors may have a much more idiosyncratic, fixed agenda. With LLMs, I have to be cautious about a small number of fixed epistemic issues that are easy to recognize with training. With humans, the trickery is much more diverse and deliberate. I feel more on my guard with human writing than with LLMs in my own chats.
That said, the LLM writing feels purely and universally disposable to me. Good human writing does not. Interacting with LLMs feels like pure information processing, while human writing feels like communication.

DirectedEvolution 18 Mar 2026 8:11 UTC
2 points
0
on: Medical Roundup #7
I honestly would not be surprised if the high cost of medical trials is the result of lobbying by big pharma companies for more red tape to stave off competition from smaller competitors.

DirectedEvolution 5 Mar 2026 20:43 UTC
5 points
0
in reply to: Raemon’s comment on: Raemon’s Shortform Feed
I don’t know that we’ve actually lowered the bar for student tier content.
Fair! Let me rephrase. LW may have historically set its quality floor low enough to permit student-tier content, because even if it’s of minor interest, it has a beneficial side effect in promoting the author’s intellectual growth and potential to contribute in a more substantial way in the future. Most content above the current quality floor reflects enough content interest and growth value to be worth accepting.
When AI slop consistently rises above that quality floor, then including it will fill LW with minor-interest slop that has no beneficial side effect of intellectual growth for the contributor. So it will become untenable to keep the floor in the same place. But LW still wants to give student-tier contributors a way to make those contributions without getting drowned out by AI slop or filtered out by a quality bar that has to keep rising to filter out the slop. The strategies I proposed are implementation ideas for that alternate submissions channel.
Is this a fair description of the problem, as you see it?

DirectedEvolution 5 Mar 2026 18:10 UTC
10 points
0
in reply to: Raemon’s comment on: Raemon’s Shortform Feed
It seems like the problem you’re articulating has to do with the fact that LessWrong functions partly as a training ground for rational thinking and AI alignment research. In the past, you’ve lowered the epistemic bar for student-tier content, because receiving feedback on their content motivates them and provides useful feedback to them.
But soon, slop-posters may meet or exceed the epistemic bar achieved by students. Judging it on a pure minimum quality standard would admit a tidal wave of useless, inert slop. It’s not good enough to contribute to the leading edge of the conversation, and it doesn’t benefit from feedback. It just drowns out the authentic student content, which really does benefit from community input.
If that accurately reflects the problem you’re concerned about, then one possibility is to enforce an escalating minimum quality bar that exceeds both the quality of AI slop and the current minimum standards of LessWrong today. Anybody can freely post, but if moderators don’t feel the quality is excellent, then it does not get any visibility on LessWrong.
Simultaneously, create a separate submission channel for “student contributions” where quality standards are lower, but the primary rationale is that the poster would benefit from community feedback to support their intellectual growth.
This strategy would introduce the question of how to set standards and vet submissions through the student channel. This may take quite a bit of thinking to strike the right balance between effectively filtering for what you want and not taking undue amounts of moderation effort. Some possible aspects of such filtering might include:
- Making default acceptance of submissions through the student channel a benefit of participating in real-world community activities, such as Inkhaven, MATS, workshops, and so on. This functions as a costly signal of a person who’s earnestly trying to use these resources to improve their capabilities. When people register for these events, they can register their account username as well, which labels that account as being permitted to make “student submissions.”
- Posting a sufficient amount of excellent content might also enable people to post through the “student submissions” channel.
- Giving the community a way to flag posts submitted through the student channel that feel AI-corrupted, with a low threshold for those posts being taken down automatically due to being flagged.

DirectedEvolution 26 Feb 2026 3:53 UTC
4 points
0
on: AllAmericanBreakfast’s Shortform
Lazy coding is a middle ground between vibe coding and traditional. Write code that may be buggy and inelegant, but clearly shows exactly what you intend to do. Then feed it into AI to fix edge cases and gold-plate it.
Writing code is a great way to think through what you’re trying to do. Prose can be a lot harder, especially when you’ve spent vastly more time expressing concepts in code (nested for loop) than you have in language (Cartesian product). AI’s also seen these ideas in code more than it has in language.
At this point, I think it’s safe to say that AI’s just better than most people at code generation. It struggles with context, and no wonder. A lot of that context is in your head, not the training data. Writing crappy but illustrative code is just a way to provide that context. You’re in a better position to vet it, having written the shitty first draft yourself.

DirectedEvolution 25 Feb 2026 20:46 UTC
2 points
0
in reply to: gwern’s comment on: AllAmericanBreakfast’s Shortform
Haven’t explored that option, thanks for the suggestion.
I agree sycophancy is part of the problem. However, the issue I’m getting at here has more to do with biases introduced by the way ideas are selected using my current approach.
Idea viability follows a multiplicative hurdle model based on independent factors. VALUE = A x B x C x D. AI (and human operator) only have intelligence and attention to address a subset of factors at one time (A and B). The others (C and D) are ignored. This problem is compounded by iteratively sampling ideas:
PARTIAL_VALUE1 = A1 x B1, PARTIAL_VALUE2 = A2 x B2, …
The problem is finally exacerbated by picking the idea with highest PARTIAL_VALUE and optimizing the in-focus factors (A and B) while refining technical execution plan focused on these factors.
Reintroducing the selected idea by itself to the AI identifies forgotten factors C and D, which contain fatal flaws simply because most untried ideas are unviable for multiple reasons in a competitive market. Selecting for ideas where in-focus factors (A and B) are viable also selects for ideas where out-of-focus factors (C and D) are unviable.

DirectedEvolution 25 Feb 2026 19:03 UTC
2 points
0
in reply to: DirectedEvolution’s comment on: AllAmericanBreakfast’s Shortform
Also, according to Gemini pro itself in response to me pasting this post into the chat, “System recommendations for Google products during execution phases occur because they are statistically dominant in enterprise operational datasets, not due to hardcoded upselling directives.”

DirectedEvolution 25 Feb 2026 19:01 UTC
9 points
0
on: AllAmericanBreakfast’s Shortform
I’ve been spending a lot of time exploring business ideas with Gemini 3.0 pro. It is both persuasive and inconsistent.
The pattern that has emerged is this:
1. I run a set of ideas, along with my goals and context, by Gemini sequentially. It rejects most, but approves one or more of them. Some ideas are my originals, others are Gemini-proposed improvements of my ideas.
2. I use Gemini to develop the idea and produce a rough business plan. It strongly encourages me to take immediate, costly steps to execute: purchasing administrating infrastructure, taking long trips to inspect leads, contacting vendors or potential customers. It suggests Google products at times, though it’s possible these simply are the best choices for the problem it’s addressing.
3. After hitting my Gemini pro usage cap, I continue using fast mode to explore “pro-approved” business ideas. Fast mode lacks judgment to evaluate ideas, but can accurately answer many questions about practical execution details.
4. The next day, when my usage limit refreshes, I summarize the previous business idea for Gemini. It immediately identifies a profound flaw in the idea that I find compelling, causing me to drop the idea.
My explanation for this pattern is that business viability is all-or-nothing: solve every problem, or fail. There are too many problems to focus on all of them in a single conversation. When the subset I can focus on gets solved by an idea, execution of the idea becomes the new focus of the conversation. But reintroducing the idea outside the previous context allows Gemini to focus on a new set of problems. Since nearly all business ideas have at least one fatal flaw, there will almost always be one for Gemini to discover.
Here are some examples of this dynamic. As a reminder, many of these ideas are Gemini-proposed modifications of ideas I’d suggested that were subsequently immediately rejected by Gemini in a followup conversation.
- Idea: Providing business intelligence mined from government sources to rural real estate developers. Fatal flaw: tells experts what they already know or devolves into a consulting gig.
- Idea: Facilitate paying owners of unlisted property to discuss a possible sale. Fatal flaw: willingness to sell is not bottlenecked by willingness to discuss a sale.
- Idea: Pay to message CEOs. Fatal flaw: already tried and failed. CEOs won’t manage a separate inbox and won’t disclose their email.
- Idea: Biotech recruiter lead sourcing from academia based on academic data exhaust. Fatal flaw: Turned from bullish to bearish on ability to extract valuable information from this exhaust.
- Idea: Healthcare admin for allied health professionals working in home settings or remotely. Fatal flaw: Unacceptable unit economics.

DirectedEvolution 25 Feb 2026 1:13 UTC
2 points
−3
on: What is a species?
Speaking as a scientist, best practice is to provide the working definition for critical terms. It would be unwise to assume that any biological definition is universal, static, comprehensive, and known to the audience. Putting the onus on the writer to disambiguate their terms also alleviates the requirement to achieve a uniform definition. Scientific categories are basically statistical in nature. All models are wrong, some are useful.
My understanding is that in legal world, there is a different mechanism for dealing with ambiguity: we empower a judge to interpret the law. Laws, in turn, are written with the understanding that they will be interpreted by judges, enforced by police, and modified by politicians elected by voters.
Your argument implies we should use political discomfort to decide which biostatistical model the term “species” refers to, then require that judges use that model when interpreting the law. I would never accept this as a scientist, I would not want judges to adopt this heuristic, and the argument basically reads to me like a bit of motivated linguistic political theater.

DirectedEvolution 24 Feb 2026 18:33 UTC
4 points
0
on: AllAmericanBreakfast’s Shortform
I’ve been confused why so many products are marketed using “with AI” as a primary selling point. First, I’m a salesperson, why do I care what technology you’re using in your lead sourcing app, as long as the leads are good? Second, the autocomplete for “AI” is “slop.” Why would you want to associate that with your product?
Now I think I understand: they’re managing user expectations down, and investor expectations up.

DirectedEvolution 24 Feb 2026 4:53 UTC
4 points
0
in reply to: Garrett Baker’s comment on: AllAmericanBreakfast’s Shortform
(occasionally)

DirectedEvolution 24 Feb 2026 4:40 UTC
2 points
0
in reply to: Brendan Long’s comment on: AllAmericanBreakfast’s Shortform
I hadn’t been following that story, but I’d heard that tokenization problems were a likely issue, so interesting to see confirmation.

DirectedEvolution 23 Feb 2026 23:26 UTC
16 points
3
on: AllAmericanBreakfast’s Shortform
One AI slop/hallucination sign I haven’t seen pointed out before are use of weasel words like “sometimes” and “occasionally.” It seems to use these when it doesn’t have an explanation for a problem and is confidently speculating. This is most noticeable when it’s giving a false diagnosis for a syntax error in a programming language (“default is a reserved keyword in Groovy. Occasionally, it may result in a syntax error if you use ‘default’ as the key in a hashmap”).
(This particular example is from Gemini fast mode. 3.0 pro recognized the true problem and proposed an effective solution).

DirectedEvolution 21 Feb 2026 10:33 UTC
0 points
0
on: AllAmericanBreakfast’s Shortform
More Gemini 3 Pro weirdness. It seems to somehow have started dumping its “show thinking“ text into the main chat output. Then it went into a loop of critiquing its own output with respect to my hardcoded universal prompt, then went into a loop of reacting to its own self-critique, finally collapsing to “I’m Outputting… Done… Bye” over and over again.

DirectedEvolution 21 Feb 2026 7:39 UTC
4 points
−5
on: AllAmericanBreakfast’s Shortform
AI improvements to date may come from picking low-hanging fruit. It can’t do math reliably? Let it use a calculator. It improves with more parameters? Scale it up and see if it helps even more.
These improvements rely on the availability of significant, well-defined problems with concrete solutions.
As these polite problems are solved, we may find that vendors find that the issues user report are increasingly hard to define, do not have clear solutions, or only have solutions that entail significant tradeoffs.
The rush by companies to deploy massive capex to scale further despite how the model training horse race destroys profitability supports this hypothesis. If they had cheap alternative ways to scale, they’d be prioritizing that. They may be out of other ideas and hoping that spending $ will lead to a magical, mysterious AI-guided ultimate victory. Hence, they call it a “scaling law,” instead of a “recently observed historic trend.”
Supply is elastic. It may also be that companies will discover new concrete and cheap dimensions of improvement as they scale, and as AI adoption continues and as production practices are reoriented around AI and anticipation of AI improvements.
But we do see true asymptotes in other industries. In transport, drivers and airline passengers are not getting around appreciably faster now than they were in 1980. The most we hope for is supersonic.
Food is much better than it was in the past, but nobody is expecting that food will eventually be ten times tastier and more nutritious than it is now due to innovation in food technology.
Shipping was revolutionized by containers and improved by guidance technologies, but again, nobody is anticipating dramatic improvements in that sector either.
Video games have improved, but it seems clear that the obvious dimensions of improvement have been mostly explored, and that innovations now appeal primarily to niche interests due to innovations in game markets and engines that enable those niche interests to be satisfied at low cost. Movies, music, art, fashion and literature have had more time to mature and are even more widely recognized as having arrived at some sort of asymptote.
Construction, an ancient industry, continues to improve slowly, with new materials and tools and manufacturing methods and regulatory practices. Perhaps because this industry is so diverse and because buildings and building methods must satisfy such a diverse range of requirements both from consumers and from producers, there are enough dimensions of improvement to keep progress going even after all this time.
Moore’s Law is slowing down. But GPUs seem to continue to undergo fast-paced innovation, and trade secrets still seem critical to the industry, which I would not expect in an industry that has saturated its space for progress.
In energy, there is clearly plenty of room for innovation, from batteries, to solar and wind, to nuclear, to fission. Slow growth in this sector in nuclear and fission seems to be primarily due to regulation and lack of research funding.
Biomedical advancements continue. Like energy, the slow pace of conversion into products is primarily attributable to regulation/political punishment, not exhaustion of the innovation search space.
We can see that while saturation of innovation opportunities is real, industries can be ancient (construction) and yet continue to find meaningful ways to innovate. They can also be novel (movies and video games) and yet exhibit signs of exhaustion. Innovation can slow due to exhaustion, regulation, or lack of investment.
It seem superficially unwise to assume a technology as novel as AI has reached saturation. However, our deeply limited theory on what “intelligence” is, and how to improve it, sets it apart from other engineering disciplines as well as from computer science generally. It puts it in some ways closer to fields like management, sociology and politics (where we struggle to define what exactly it is we’re studying) and also to fields like biology (where mechanisms are complex and hard to tease apart, but where we can at least try to run controlled experiments and have at least some theory to guide their design).
My guess is that progress in AI will follow a trajectory analogous to progress in medicine. There was a sharp break in the late 1800s as concepts of sanitation and germ theory and chemistry took hold, leading to a sudden and rapid picking of low hanging fruit, a general excitement about the possibilities of progress, an embrace of the “radical,” a public fascination.
Then the low hanging fruit was exhausted. Despite the continuous and extraordinary improvements in methods and understanding that could have been seen as trending toward “self-improvement” and acceleration and “takeoff,“ this isn’t what we see.
Instead, limited ability to probe, predict and intervene on complex biological systems over relevant time and population scales has made every inch of progress hard won. We cannot just experiment on people. We have struggled to produce quality models. We have discovered one dimension of profound challenge after another, as hype cycle has followed hype cycle in the quest for immortality, a quest that’s just as grandiose as the quest for AGI.
AGI will face its own analogous challenges. It is so expensive to train and run models that you can’t freely and infinitely run useful experiments on this process. We lack the equivalent for intelligence of germ theory, evolution, chemistry and anatomy: we can’t break down intelligence into component parts to understand their function and relationship to the whole. We cannot identify the equivalent of “the DNA of intelligence“ or explain hallucinations the way we can explain heart attacks.
AI innovation has advantages that medicine lacks: it is operating on machines, so there are fewer ethical and legal barriers to experiment, and it is running on hardware and software that can be directly inspected and precisely modified down to the level of bits.
It’s unclear to me how the advantages and disadvantages of AI will play out in terms of progress rates over time. I‘m confident AI is not at some fundamental asymptote of possible progress. But every other industry I can think of has transitioned from an early boom in innovation to a long grinding slow climb in capabilities over a century or more.
i understand the objection that unlike other technologies, which are capital, AI can act as and improve on and replace labor. That the capital/labor division is not a fundamental truth, just categories of convenience that have exhausted their utility.
But that’s not really what I’m arguing against. I just think it might still take a long time to go from where AI is now to it having the full complement of human capabilities that would make it a genuine, general substitute for human labor. We don’t actually know what those capabilities are. We are actively discovering our differentiators as we go.
In the past, we thought that certain capabilities like artistic creativity or chess were distinctively human. We’ve seen AI demolish those fantasies.
But we have also seen that human capabilities a computer could easily replicate, like demonstrating awareness of what it does not know, or holding a consistent opinion, or recognizing when you’re going in circles, are actually difficult for AI. And we don’t know why. Only with advancing capabilities do we discover what’s truly easy or hard for a computer.
We also haven’t adequately considered the possibility that as capabilities advance, the capabilities requirements to produce further gains may also advance. If it takes a team of brilliant world-class human AI experts to produce current marginal AI capabilities gains, what if there comes a pre-full-AGI day when neither they nor whatever the frontier models are at that time are capable of eking out substantial further improvements in capabilities? It may be too hard for humans plus subhuman AI to make a human-level AI.
This is only a hypothesis, but it leaves some real room for doubt as to where AI will be in the next 30 or even 100 years. I might very well be an old man who lives in a world full of exceptional AI-produced software tools for processing information and where factories or corporations are managed autonomously by AI, and yet which resembles this world about as much as this world resembles 1975.

DirectedEvolution 16 Feb 2026 11:44 UTC
3 points
0
in reply to: Lucie Philippon’s comment on: A multi-level postmortem of how our whole house got badly poisoned
By experiment, I guess? I don’t have a well worked out philosophy on this specific issue. But in general, I think that when you have a plausible idea like this, it’s good to just budget some time for a few tests to learn how onerous it is (or isn’t), and see if it catches anything that make it seem worthwhile. You’ll probably figure out intuitively when the amount of inventorying and investigation is starting to seem excessive. And probably the level you should be at is not zero. So then it’s just about figuring out the highest priority areas and working your way down the list gradually over time.
I’ll bet you could figure out a solid plan (especially now with AI) for inventorying potential safety issues in your house in an afternoon and execute in a few hours, especially with housemates to help.

DirectedEvolution 15 Feb 2026 6:04 UTC
3 points
0
on: Why You Don’t Believe in Xhosa Prophecies
What does it mean for ideas to “replicate on AI substrate“ that makes this a specific, interesting phenomenon?

DirectedEvolution 15 Feb 2026 1:47 UTC
7 points
0
on: A multi-level postmortem of how our whole house got badly poisoned
I like this article. Sorry you ran into such a scary situation!
This may seem like a tenuous connection, but my fiancé and I are planning our wedding, and this article made me think about how we’re approaching that process. We both worked on inventorying the details and risks we needed to quiz venues about—surprise upcharges, ADA accessibility, the level of service, whether they had vendors experienced with their venue and willing to continue working there.
It didn’t take too long to master a pretty comprehensive list of questions, and of course we can write it down too. This lets us pretty efficiently vet places both by email/phone and in person. I’m by no means a naturally suspicious person, but I find that applying that scrutiny comes easily when I prepare the checklist of questions to ask in advance.
Probably the same approach can be applied to other big decisions, including renting or buying a home, choosing a romantic partner, picking a career, school or employer. I find approaching these questions systematically makes me feel more relaxed and in control and interested in the process. AI is very helpful for brainstorming the questions to ask.
So just a generally more structured and systematic approach to decision making seems good. Or for decisions you’ve already made (ie the house you’re already in), making a habit of a periodic, maybe annual, inspection or inventory. Seems like a very healthy habit to get into.

DirectedEvolution 14 Feb 2026 6:44 UTC
2 points
0
in reply to: Caleb Biddulph’s comment on: CBiddulph’s Shortform
And predictably, despite daily coverage of niche AI topics, not a word about in from Tyler Cowen on Marginal Revolution.