azergante

Karma: 122

azergante Aug 1, 2025, 10:23 AM
7 points
0
on: Why I Reject the Correspondence Theory of Truth
You claim:
1. We do not have direct access to the world.
2. Our access to the external world is entirely mediated by models
3. Pragmatic evaluation, on the other hand, is world-involving. You’re testing your models against the world, seeing how effective they are at helping you accomplish your goal.
If you claim that (1) we do not have direct access to the world and that (2) access to the world is mediated through models then you also need to explain how (3) pragmatism allows us to test our models against the world, and you need to explain it in terms of (2) since models are the only mediator to the world.

I don’t think you give a satisfactory explanation for that, possibly a key is to precisely define what you mean by “world”. Given (1) and (2) I think that if you posit an external world it needs to be defined in terms of (2).

Note that I am not agreeing or disagreeing about the truth of 1), 2) and 3), just pointing out a contradiction or a missing explanation.

My stab at defining “world”:

a) we make observations
b) we create mathematical models of those observations
c) what we call “world” is actually a logical object defined by the widest possible application of all our mathematical models

In this view we only need to make sure that our models match our observations so the correspondence theory of truth is fine, however the “territory” or world turns out to be a super-model which I think is a significant departure from the usual map-territory distinction.

azergante Jul 30, 2025, 11:36 AM
3 points
0
on: My Empathy Is Rarely Kind
To improve, you may want to start by sketching out what an ideal interaction with a person that has a nail in their head looks like for you, and figure out how to get closer to that.

To me such an ideal interaction could be:
1. removing the disgust (because it has low valence)
2. feeling at ease with the fact that there are people in the world that have nails in their head (remembering that you, them and the nail are the natural unfolding of physics might help)
3. feeling joy when people (or myself) improve (the joy keeps the incentive to help them and become stronger myself)
I think the gist is removing low valence internal events and replacing them with high valence ones, while keeping the incentives to be functional.

I am not sure how much it’s possible to shift on the valence axis while retaining functionality (given the human reward circuitry) but some people do look much happier than others (be it because of genetics, meditation or other) and they usually say it makes them more productive so I’m rather optimistic.

azergante Jul 28, 2025, 1:39 PM
1 point
−2
on: Taking Abundance Seriously

Imagine pushing and pushing and realising you couldn’t effect any change.

Eliezer actually addresses this in Free to Optimize

if there’s an AI that really does look over the alternatives before I do, and really does choose the outcome before I get a chance, then I’m really not steering my own future. The future is no longer counterfactually dependent on my decisions.

And much more about some of the traits a utopia that is actually fun to live in would have in his Fun Theory sequence. It is quite an interesting read.

azergante Jul 23, 2025, 9:57 PM
3 points
0
on: Comment on “Four Layers of Intellectual Conversation”
Paradoxically the link to the archive only provides L0 (thesis), and no L1-3 (critique, counter-critique, counter-response). This is unfortunate because Eliezer manages to criticize priests, chemists, academic reviewers, mainstream media and EAs all in one post, and it would have been interesting to have their take too.

As far as I can tell the post backs up its thesis with anecdotal evidence but no quantitative data, so let’s not update too much.

About “maybe not X”:
- Maybe these deep back and forth happen more than Eliezer thinks?
- Maybe we have limited criticism capacity and it’s already all used up on high priority stuff?
- Maybe most people don’t have anything meaningful to add so telling them to criticize more isn’t useful?

azergante Jul 18, 2025, 8:24 PM
1 point
0
in reply to: Viliam’s comment on: Why is LW not about winning?

Sure, agency and power are good. If you think there is a low-hanging fruit we should pick, please explain more specifically.

I cannot be more specific about winning rationality because I don’t know how to do it. One would first have to set out to create the art, go out there, win and report back.

Agency, we have discussed a lot already (1, 2, 3),

Power is a zero-sum game

Then again I might read more of what people have published on LW and find that it’s already as good as it gets, who knows.

azergante Jul 16, 2025, 6:14 PM
12 points
7
in reply to: Loki zen’s comment on: Loki zen’s Shortform
Maybe start with 3Blue1Brown series on Neural networks ? This is still math but it has great visualizations.

azergante Jul 15, 2025, 5:12 PM
2 points
0
in reply to: Raemon’s comment on: Why is LW not about winning?
My above comment (not focusing on the main post for a moment) does not claim that it’s easy to hire alignment researchers, but that “you can’t use money to hire experts because you can’t reliably identify them” is the wrong causal model to explain why hiring for alignment is difficult because it’s false: if that causal model were true, you’d expect no companies to be able to hire experts, which is not the case. Anyway, maybe this is nitpicking but to me something like “AI alignment is in its infancy so it’s harder to hire for it than for other fields” would be more convincing.
your initial post was built on a mistaken premise
I do miss a lot of background on what has been discussed and tried so far, in retrospect most of what I read on LW so far is Rationality: A-Z and the Codex, plus some of the posts in my feed.
If the library had a “A Short History of AI alignment” section I probably would have read it, maybe pinning something like that somewhere visible will help new users get up to speed on the subject more reliably? I do understand that this is a big time investment though

azergante Jul 14, 2025, 8:41 PM
6 points
0
in reply to: johnswentworth’s comment on: Why is LW not about winning?
I read both of the posts you link to, I interpret the main claim as “you can’t use money to hire experts because you can’t reliably identify them”.

But the reality is that knowledge companies do manage to hire experts and acquire expertise. This implies that alignment research organizations should be able to do the same and I think it’s enough to make the the strong version of the claim irrelevant.

I agree with a weaker version which is that some amount of money is wasted because hiring is unreliable, but again it’s the same for all knowledge companies and society has many mechanisms such as reputation, diplomas and tests to better navigate these issues already.

Edit: your argument about Jeff Bezos rings very wrong to me

Last I heard, Jeff Bezos was the official richest man in the world. He can buy basically anything money can buy. But he can’t buy a cure for cancer. Is there some way he could spend a billion dollars to cure cancer in five years? Probably, yes. But Jeff Bezos does not know how to do that. Even if someone somewhere in the world does know how to turn a billion dollars into a cancer cure in five years, Jeff Bezos cannot distinguish that person from the thousands of other people who claim to know (and may even believe it themselves) but are wrong.

I was not sure how to articulate that so I pasted the 1st paragraph of the essay into Claude and I think it did a good job of pointing deficiencies, here is the full transcript:
LLM transcript
Me: evaluate that argument with critical thinking “”″First Puzzle Piece By and large, the President of the United States can order people to do things, and they will do those things. POTUS is often considered the most powerful person in the world. And yet, the president cannot order a virus to stop replicating. The president cannot order GDP to increase. The president cannot order world peace. Are there orders the president could give which would result in world peace, or increasing GDP, or the end of a virus? Probably, yes. Any of these could likely even be done with relatively little opportunity cost. Yet no president in history has known which orders will efficiently achieve these objectives. There are probably some people in the world who know which orders would efficiently increase GDP, but the president cannot distinguish them from the millions of people who claim to know (and may even believe it themselves) but are wrong. Last I heard, Jeff Bezos was the official richest man in the world. He can buy basically anything money can buy. But he can’t buy a cure for cancer. Is there some way he could spend a billion dollars to cure cancer in five years? Probably, yes. But Jeff Bezos does not know how to do that. Even if someone somewhere in the world does know how to turn a billion dollars into a cancer cure in five years, Jeff Bezos cannot distinguish that person from the thousands of other people who claim to know (and may even believe it themselves) but are wrong. When non-experts cannot distinguish true expertise from noise, money cannot buy expertise. Knowledge cannot be outsourced; we must understand things ourselves.”″”

Claude: This argument presents an interesting paradox about power and knowledge, but it contains several logical flaws and oversimplifications that weaken its conclusions.

Strengths of the argument: The core insight is valuable—that raw power (political authority, wealth) doesn’t automatically translate into the ability to solve complex problems. The distinction between having resources and knowing how to deploy them effectively is important. The argument correctly identifies that information asymmetry creates genuine challenges for decision-makers.

Critical weaknesses:

False dichotomy on expertise identification: The argument assumes leaders must either perfectly identify true experts or remain completely helpless. In reality, there are established methods for evaluating expertise: peer review, track records, institutional credentials, reproducible results, and consensus among qualified professionals. While imperfect, these mechanisms work reasonably well—we don’t see random people performing surgery or designing bridges.

Ignoring existing institutional solutions: The argument overlooks how real-world leaders actually operate. Presidents don’t personally evaluate every expert claim—they rely on established institutions (CDC, Federal Reserve, Pentagon) that have developed systematic approaches to vetting expertise. Bezos doesn’t personally screen cancer researchers—he funds established institutions and relies on scientific peer review processes.

Overstated claims about knowledge and outsourcing: The conclusion that “knowledge cannot be outsourced” is demonstrably false. We successfully outsource knowledge constantly—when you visit a doctor, fly in an airplane, or use a smartphone, you’re benefiting from outsourced expertise. The key is having reliable mechanisms for credentialing and accountability.

Survivorship bias in examples: The argument cherry-picks unsolved problems (cancer cure, world peace) while ignoring areas where wealth and power have successfully purchased expertise to solve complex problems—like developing COVID vaccines, creating global supply chains, or landing on the moon.

Conflating different types of problems: The argument treats technical problems (virus replication), economic problems (GDP growth), and political problems (world peace) as equivalent, when they have fundamentally different characteristics and solution pathways.

Better framework: Rather than concluding that power is useless without perfect knowledge, a more nuanced view would recognize that effective leadership involves building systems and institutions that can reliably identify and deploy expertise, while acknowledging the inherent uncertainties and limitations in complex problem-solving.

The argument raises important questions about the relationship between power and knowledge, but its binary thinking and sweeping conclusions undermine its analytical value.

azergante Jul 14, 2025, 11:02 AM
4 points
0
in reply to: johnswentworth’s comment on: Why is LW not about winning?

It turns out that, no, hiring lots of other people is not actually how you win when the problem is hard.

Using money to hire people is only one strategy among many, if it does not work then maybe starting an alignment course at your local university does? Maybe talking to your mayor is useful? what about becoming the mayor? but how would we know since LW mostly does not talk about that?

These are just examples, surely among all the possible strategies we can find a few that are worth doing and give leverage over direct research? (LessWrong itself would be an example). This seems worth exploring.

azergante Jul 14, 2025, 10:49 AM
4 points
0
in reply to: Raemon’s comment on: Why is LW not about winning?
Note: I have not read the linked posts yet, will do that later
a) I don’t see many posts to the tune of “What do you think of [some strategy that doesn’t involve direct research on alignment]?” (maybe getting influence in your local town hall, or university etc), perhaps you can point out to such posts? In the same way I don’t see a lot of experience reports like “I paused alignment research and went off this other route instead, hoping for an efficiency multiplier. Here’s what worked and here is what didn’t”.
I am not saying these posts never happen, but given the possible leverage, I would expect to see more of them. I think it’s fair to say that there are a lot more posts about direct research than about other (leveraged) ways to approach the issue. For example here is my LW feed, there are 3.5 posts about alignment (highlighted), 3.5 about AI and none about other strategies (the post “Lessons from the Iraq War for AI policy” is still pretty far from that as it does not discuss something like a career path or actions that can be taken by an individual).
You say these have happened a lot, but I don’t see this discussed much on LW. LW itself can be characterized as Eliezer’s very successful leveraged strategy to bring more people into alignment research, so maybe the leveraged strategies end up discussed more outside LW? But in any case this at least shows that some leveraged strategies work, so maybe it’s worth discussing more.
b) I think this can be summarized as “we don’t know how to put more resources into alignment without this having (sometimes very) negative unintended outcomes”. Okay fair enough, but this seems like a huge issue and maybe there should be more posts about exploring and finding leveraged strategies that won’t backfire. Same for power seeking, there is a reason why power is an instrumental goal of ASI, it’s because it’s useful to accomplish any goal, so it’s important to figure out good ways to get and use power.
Now maybe your answer is something like “we tried, it didn’t work out that well so we re-prioritized accordingly”. But it’s not obvious to me that we shouldn’t try more and develop a better map of all the available options. Anyway, I will read up on what you linked, if you have more links that you think would clarify what was tried and what worked/didn’t work don’t hesitate to share.

[Question] Why is LW not about winning?

azerganteJul 13, 2025, 10:36 PM

21 points

21 comments1 min readLW link

azergante Jul 13, 2025, 8:53 PM
1 point
0
on: Rationality !== Winning

Rationality !== Winning

I do wish there were more LessWrong content focused on how to win, for real.

I mean, LessWrong users seem to care a lot about alignment, but if you really want to solve that problem, and you want to be efficient about it, it seems super obvious that there are better strategies than researching the problem yourself, like don’t spend 3+ years on a PhD but instead get 10 other people to work on the issue ; and that 10x s your efficiency already.

My point is that thinking about it rationally, it looks like there are ways to get orders of magnitude more progress done on alignment by doing something that is not researching the problem yourself.

So I would expect a LW focused on winning to talk a lot more about agency and power, and how to use these for good.

azergante Jul 13, 2025, 8:18 PM
1 point
0
in reply to: Raemon’s comment on: Rationality !== Winning

typically ”!=” means “X not equal to Y” and ”!==” means “X not exactly equal to Y”

This is a tangent but PHP & Javascript are the only 2 programming languages that have a !== operator (in the top 20 languages by users), and they implement this extra operator to disable the type conversions automatically performed by !=.

Common wisdom is that these automatic conversions are a source of bugs and having them on the shorter, widespread != is bad design. All this to conclude that when I see !== it does trigger a slight eww, bad reaction.

azergante Jul 8, 2025, 9:55 PM
1 point
1
on: Foom & Doom 1: “Brain in a box in a basement”

1.3 A far-more-powerful, yet-to-be-discovered, “simple(ish) core of intelligence

1.3.1 Existence proof: the human cortex

The brain is not simple and I don’t expect to find it simple once we understand how it works.

There is an incoherence in these sections: you justify the existence of a “core of intelligence” simpler than LLMs by pointing at brains that are messier than LLMs.

azergante Jul 2, 2025, 9:45 PM
1 point
0
on: Counterarguments to the basic AI x-risk case
FWIW this post made me update in favor of AI X-risk, as I had not read counterarguments until now and expected stronger ones.

azergante Jun 23, 2025, 7:36 PM
13 points
2
on: Consider chilling out in 2028
How about we learn to smile while saving the world? Saving the world doesn’t strike me as strictly incompatible with having fun, so let’s do both? :)

The post proposes that LessWrong could tackle alignment in more skillfull ways which is a wholesome thought, but I feel that the post also casts doubt on the project of alignment itself ; I want to push back on that.

It won’t become less important to prevent the creation of harmful technologies in 2028, or in any year for that matter. Timelines and predictions don’t feel super relevant here.

We know that AGI can be dangerous if created without proper understanding and that fact does not change with time or timelines, so LW should still aim for:
1. An international framework that restricts AGI creation and ensures safety, just like for other large impact technologies
2. Alignment research to eventually reap the benefits of aligned AGI, but with less pressure as long as point 1 stands
If the current way of advancing towards the goal is sub-optimal, giving up on the goal is not the only answer, we can also change the way we go about it. Since getting AGI right is important, not giving up and changing the way we go about it seems like the better option (all this predicated on the snob and doommish depictions in the post being accurate).

azergante Jun 17, 2025, 6:51 PM
1 point
5
on: azergante’s Shortform
A paperclip maximizer would finally turn itself into paperclips after having paperclipped the entire universe.

azergante Jun 15, 2025, 1:58 PM
1 point
0
on: Incremental Progress and the Valley
Insofar as this is an issue, it would be useful to have a map of the stages of the progress of rationality, so we can put common valleys on the map and avoid them (this is inspired by some meditation traditions that have maps of meditation stages one of which is called the “Dark Night”).

I’m talking about how startup founders strike it rich by believing in themselves and their ideas more strongly than any reasonable person would.

For each founder that succeeds this way how many waste years in a doomed venture because they believe too much in themselves, how many would have avoided this trap by being well-calibrated rationalists?

My theist girlfriend, with whom I was very much in love, couldn’t deal with this change in me

In this example the commenter’s environment assigns negative value to rationality, it’s worth clarifying that his problem-solving/winning ability did not go down though.

My take-away from the examples is that if I am deceiving myself into thinking the world is better than it actually is (or just unaware) it’s likely my happiness levels will go down somewhat when I find out the truth, conversely it’s worth noticing that if I think the world is worse than it actually is (e.g. paranoia) then my happiness will go up as I find out the truth.

azergante Jun 14, 2025, 2:25 PM
1 point
0
on: Helpless Individuals
Funding large-scale projects seems like a perfect job for financial markets. Maybe society should figure out the detailed reasons why financial markets don’t invest (more) in science, and fix these issues, one step at a time.

I suspect allocation of returns is one such issue, but it looks solvable by having shares over patents, risk mitigation should work by diversifying the fund, same as current index funds. Diversification also mitigates the long-term/spiky nature of the investment, but if sporadic returns are still an issue they could be smoothed out by an insurance (it may be less of an issue if this is a long-term investment for retirement for example?).

Edit: or simpler: have the government issue bonds for funding (specific fields of) science, the government pays out coupons based on how much the bond is expected to increase GDP. The money to pay the coupons comes from the tax revenue increase generated by the (expected) GDP increase generated by the science bond sell.

Related: Social Impact Bonds

azergante Jun 10, 2025, 8:56 PM
1 point
0
on: Shut up and do the impossible!
Did Eliezer ever played the human part in these experiments?

When someone can’t think of any possible argument that would convince them of something, that just means their brain is running a search that hasn’t yet turned up a path.

I wonder if he already knows what sequence of words would cause him to let the AI out?

azergante

[Question] Why is LW not about win­ning?

[Question] Why is LW not about winning?