leogao

Karma: 5,433

leogao Feb 24, 2025, 2:48 AM
2 points
0
in reply to: Thomas Kwa’s comment on: leogao’s Shortform
i’m happy to grant that the 0.1% is just a fermi estimate and there’s a +/- one OOM error bar around it. my point still basically stands even if it’s 1%.
i think there are also many factors in the other direction that just make it really hard to say whether 0.1% is an under or overestimate.
for example, market capitalization is generally an overestimate of value when there are very large holders. tesla is also a bit of a meme stock so it’s most likely trading above fundamental value.
my guess is most things sold to the public sector probably produce less economic value per $ than something sold to the private sector, so profit overestimates value produced
the sign on net economic value of his political advocacy seems very unclear to me. the answer depends strongly on some political beliefs that i don’t feel like arguing out right now.
it slightly complicates my analogy for elon to be both the richest person in the us and also possibly the most influential (or one of). in my comment i am mostly referring to economic-elon. you are possibly making some arguments about influentialness in general. the problem is that influentialness is harder to estimate. also, if we’re talking about influentialness in general, we don’t get to use the 0.1% ownership of economic output as a lower bound of influentialness. owning x% of economic output doesn’t automatically give you x% of influentialness. (i think the majority of other extremely rich people are not nearly as influential as elon per $)

leogao Feb 24, 2025, 2:29 AM
23 points
0
on: leogao’s Shortform
you might expect that the butterfly effect applies to ML training. make one small change early in training and it might cascade to change the training process in huge ways.
at least in non-RL training, this intuition seems to be basically wrong. you can do some pretty crazy things to the training process without really affecting macroscopic properties of the model (e.g loss). one very well known example is that using mixed precision training results in training curves that are basically identical to full precision training, even though you’re throwing out a ton of bits of precision on every step.

leogao Feb 24, 2025, 12:50 AM
15 points
0
on: leogao’s Shortform
there’s an obvious synthesis of great man theory and broader structural forces theories of history.
there are great people, but these people are still bound by many constraints due to structural forces. political leaders can’t just do whatever they want; they have to appease the keys of power within the country. in a democracy, the most obvious key of power is the citizens, who won’t reelect a politician that tries to act against their interests. but even in dictatorships, keeping the economy at least kind of functional is important, because when the citizens are starving, they’re more likely to revolt and overthrow the government. there are also powerful interest groups like the military and critical industries, which have substantial sway over government policy in both democracies and dictatorships. many powerful people are mostly custodians for the power of other people, in the same way that a bank is mostly a custodian for the money of its customers.
also, just because someone is involved in something important, it doesn’t mean that they were maximally counterfactually responsible. structural forces often create possibilities to become extremely influential, but only in the direction consistent with said structural force. a population that strongly believes in foobarism will probably elect a foobarist candidate, and if the winning candidate never existed, another foobarist candidate would have won. winning an election always requires a lot of competence, but no matter how competent you are, you aren’t going to win on an anti-foobar platform. the sentiment of the population has created the role of foobarist president for someone foobarist to fill.
this doesn’t mean that influential people have no latitude whatsoever to influence the world. when we’re looking at the highest tiers of human ability, the efficient market hypothesis breaks down. there are so few extremely competent people that nobody is a perfect replacement for anyone else. if someone didn’t exist, it doesn’t necessarily mean someone else would have stepped up to do the same. for example, if napoleon had never existed, there might have been some other leader who took advantage of the weakness of the Directory to seize power, but they likely would have been very different from napoleon. great people still have some latitude to change the world orthogonal to the broader structural forces.
it’s not a contradiction for the world to be mostly driven by structural forces, and simultaneously for great people to have hugely more influence than the average person. in the same way that bill gates or elon musk are vastly vastly wealthier than the median person, great people have many orders of magnitude more influence on the trajectory of history than the average person. and yet, the richest person is still only responsible for 0.1%* of the economic output of the united states.
*\ fermi estimate, taking musk’s net worth and dividing by 20 to convert stocks to flows, and comparing to gdp. caveats apply based on interest rates and gdp being a bad metric. many assumptions involved here.

leogao Feb 23, 2025, 9:58 PM
42 points
7
on: leogao’s Shortform
there are a lot of video games (and to a lesser extent movies, books, etc) that give the player an escapist fantasy of being hypercompetent. It’s certainly an alluring promise: with only a few dozen hours of practice, you too could become a world class fighter or hacker or musician! But because becoming hypercompetent at anything is a lot of work, the game has to put its finger on the scale to deliver on this promise. Maybe flatter the user a bit, or let the player do cool things without the skill you’d actually need in real life.
It’s easy to dismiss this kind of media as inaccurate escapism that distorts people’s views of how complex these endeavors of skill really are. But it’s actually a shockingly accurate simulation of what it feels like to actually be really good at something. As they say, being competent doesn’t feel like being competent, it feels like the thing just being really easy.

leogao Feb 5, 2025, 4:19 AM
77 points
10
on: leogao’s Shortform
when i was new to research, i wouldn’t feel motivated to run any experiment that wouldn’t make it into the paper. surely it’s much more efficient to only run the experiments that people want to see in the paper, right?
now that i’m more experienced, i mostly think of experiments as something i do to convince myself that a claim is correct. once i get to that point, actually getting the final figures for the paper is the easy part. the hard part is finding something unobvious but true. with this mental frame, it feels very reasonable to run 20 experiments for every experiment that makes it into the paper.

leogao Jan 31, 2025, 9:48 PM
46 points
8
on: leogao’s Shortform
libraries abstract away the low level implementation details; you tell them what you want to get done and they make sure it happens. frameworks are the other way around. they abstract away the high level details; as long as you implement the low level details you’re responsible for, you can assume the entire system works as intended.
a similar divide exists in human organizations and with managing up vs down. with managing up, you abstract away the details of your work and promise to solve some specific problem. with managing down, you abstract away the mission and promise that if a specific problem is solved, it will make progress towards the mission.
(of course, it’s always best when everyone has state on everything. this is one reason why small teams are great. but if you have dozens of people, there is no way for everyone to have all the state, and so you have to do a lot of abstracting.)
when either abstraction leaks, it causes organizational problems—micromanagement, or loss of trust in leadership.

leogao Jan 25, 2025, 4:51 AM
3 points
4
in reply to: Archimedes’s comment on: Mechanisms too simple for humans to design
the laws of physics are quite compact. and presumably most of the complexity in a zygote is in the dna.

leogao Jan 25, 2025, 4:45 AM
8 points
3
on: leogao’s Shortform
a thriving culture is a mark of a healthy and intellectually productive community / information ecosystem. it’s really hard to fake this. when people try, it usually comes off weird. for example, when people try to forcibly create internal company culture, it often comes off as very cringe.

leogao Jan 22, 2025, 8:52 AM
37 points
10
on: leogao’s Shortform
don’t worry too much about doing things right the first time. if the results are very promising, the cost of having to redo it won’t hurt nearly as much as you think it will. but if you put it off because you don’t know exactly how to do it right, then you might never get around to it.

leogao Jan 21, 2025, 9:46 PM
61 points
39
in reply to: Roman Malov’s comment on: Daniel Kokotajlo’s Shortform
the tweet is making fun of people who are too eager to do something EMPIRICAL and SCIENTIFIC and ignore the pesky little detail that their empirical thing actually measures something subtly but importantly different from what they actually care about

leogao Jan 21, 2025, 9:42 PM
17 points
11
in reply to: leogao’s comment on: meemi’s Shortform
i’ve changed my mind and been convinced that it’s kind of a big deal that frontiermath was framed as something that nobody would have access to for hillclimbing when in fact openai would have access and other labs wouldn’t. the undisclosed funding before o3 launch still seems relatively minor though

leogao Jan 21, 2025, 8:15 PM
104 points
10
in reply to: Daniel Kokotajlo’s comment on: Daniel Kokotajlo’s Shortform
lol i was the one who taped it to the wall. it’s one of my favorite tweets of all time

leogao Jan 19, 2025, 3:56 AM
14 points
−14
in reply to: meemi’s comment on: meemi’s Shortform
this doesn’t seem like a huge deal

leogao Jan 14, 2025, 4:49 AM
LW: 18 AF: 9
6
AF
in reply to: Thane Ruthenis’s comment on: Thane Ruthenis’s Shortform
in retrospect, we know from chinchilla that gpt3 allocated its compute too much to parameters as opposed to training tokens. so it’s not surprising that models since then are smaller. model size is a less fundamental measure of model cost than pretraining compute. from here on i’m going to assume that whenever you say size you meant to say compute.
obviously it is possible to train better models using the same amount of compute. one way to see this is that it is definitely possible to train worse models with the same compute, and it is implausible that the current model production methodology is the optimal one.
it is unknown how much compute the latest models were trained with, and therefore what compute efficiency win they obtain over gpt4. it is unknown how much more effective compute gpt4 used than gpt3. we can’t really make strong assumptions using public information about what kinds of compute efficiency improvements have been discovered by various labs at different points in time. therefore, we can’t really make any strong conclusions about whether the current models are not that much better than gpt4 because of (a) a shortage of compute, (b) a shortage of compute efficiency improvements, or (c) a diminishing return of capability wrt effective compute.

leogao Jan 8, 2025, 7:51 PM
5 points
0
in reply to: tangerine’s comment on: Disagreement on AGI Suggests It’s Near
suppose I believe the second coming involves the Lord giving a speech on capitol hill. one thing I might care about is how long until that happens. the fact that lots of people disagree about when the second coming is doesn’t mean the Lord will give His speech soon.
similarly, the thing that I define as AGI involves AIs building Dyson spheres. the fact that other people disagree about when AGI is doesn’t mean I should expect Dyson spheres soon.

leogao Jan 8, 2025, 7:03 AM
6 points
6
on: Disagreement on AGI Suggests It’s Near
people disagree heavily on what the second coming will look like. this, of course, means that the second coming must be upon us

leogao Jan 7, 2025, 2:36 AM
3 points
0
in reply to: Sheikh Abdur Raheem Ali’s comment on: Reasons for and against working on technical AI safety at a frontier AI lab
I agree that labs have more compute and more top researchers, and these both speed up research a lot. I disagree that the quality of responses is the same as outside labs, if only because there is lots of knowledge inside labs that’s not available elsewhere. I think these positive factors are mostly orthogonal to the quality of software infrastructure.

leogao Jan 5, 2025, 7:07 PM
39 points
15
on: Reasons for and against working on technical AI safety at a frontier AI lab
some random takes:
- you didn’t say this, but when I saw the infrastructure point I was reminded that some people seem to have a notion that any ML experiment you can do outside a lab, you will be able to do more efficiently inside a lab because of some magical experimentation infrastructure or something. I think unless you’re spending 50% of your time installing cuda or something, this basically is just not a thing. lab infrastructure lets you run bigger experiments than you could otherwise, but it costs a few sanity points compared to the small experiment. oftentimes, the most productive way to work inside a lab is to avoid existing software infra as much as possible.
- I think safetywashing is a problem but from the perspective of an xrisky researcher it’s not a big deal because for the audiences that matter, there are safetywashing things that are just way cheaper per unit of goodwill than xrisk alignment work—xrisk is kind of weird and unrelatable to anyone who doesn’t already take it super seriously. I think people who work on non xrisk safety or distribution of benefits stuff should be more worried about this.
- this is totally n=1 and in fact I think my experience here is quite unrepresentative of the average lab experience, but I’ve had a shocking amount of research freedom. I’m deeply grateful for this—it has turned out to be incredibly positive for my research productivity (e.g the SAE scaling paper would not have happened otherwise).

leogao Dec 31, 2024, 5:36 AM
2 points
0
in reply to: Nina Panickssery’s comment on: leogao’s Shortform
I think this is probably true of you and people around you but also you likely live in a bubble. To be clear, I’m not saying why people reading this should travel, but rather what a lot of travel is like, descriptively.

leogao Dec 30, 2024, 12:31 AM
19 points
2
on: leogao’s Shortform
theory: a large fraction of travel is because of mimetic desire (seeing other people travel and feeling fomo / keeping up with the joneses), signalling purposes (posting on IG, demonstrating socioeconomic status), or mental compartmentalization of leisure time (similar to how it’s really bad for your office and bedroom to be the same room).

this explains why in every tourist destination there are a whole bunch of very popular tourist traps that are in no way actually unique/comparatively-advantaged to the particular destination. for example: shopping, amusement parks, certain kinds of museums.