williawa

Karma: 27

williawa 19 Mar 2025 10:11 UTC
1 point
0
on: Feedback loops for exercise (VO2Max)
Why not just use resting heartrate? That also has very good empirical backing as a good proxy for overall heatlh, and its much easier to measure.

williawa 16 Mar 2025 11:54 UTC
1 point
0
in reply to: plex’s comment on: ete’s Shortform
I basically agree with this.
Or I’d put 20% chance on us being in the worlds “where superalignment doesn’t require strong technical philosophy”, that’s maybe not very low.
Overall I think the existance of Anthropic is a mild net positive, and the only lab for which this is true (major in the sense of building frontier models).
“the existence of” meaning, if they shut down today or 2 years ago, it would’ve not increased our chance of survival, maybe lowered it.
I’m also somewhat more optimistic about the research they’re doing helping us in the case where alignment is actually hard.

williawa 12 Mar 2025 17:11 UTC
2 points
0
on: Trojan Sky
Awesome story! Don’t have any big brain takes about what it “means”, but I like the moment-to-moment descriptions of the world and the descriptions of the mental episodes the main guy is having towards the end. It flows well.

williawa 8 Mar 2025 10:55 UTC
13 points
2
in reply to: Sam Marks’s comment on: So how well is Claude playing Pokémon?
I’m not sure. I remember playing a bunch of games, like pokemon heart gold, lego starwars, and some other pokemon game where you were controlling little pokemon in 3rd person instead of controlling a human who threw pokeballs (anyone know that game? )
And like, I didn’t speak English when I played them. So I had to figure out everything by just pressing random buttons and seeing responses. And this makes it a lot more difficult. Like I could open my “inventory” (didn’t know what that was) and then use a “healing potion” (didn’t know what that was), and then because my pokemon was at full health already, I would think the healing potion was useless, or think that items in inventory only cause text to appear on the screen, but that they don’t have any effect on the actaul, and then I’d believe this until I accidentally clicked the inventory and randomly saw a change, or had failed a level so many times that I was getting desperate and just manually doing exhaustive search over all the actions.
But like, I’m very confident I was more action efficient than claude is. Mostly because like, if I enter a battle, and like fail 5 times more or less in the same way, you start to think something is awry, and start doing different stuff. And also just because, certain things become automatic after a short while, like moving around. For claude it takes the same amount of time each time. So if you’re failing at a specific point in a battle, the fact that that point is responsible for you overall failing to progress, becomes very obvious, because anything other than that becomes automatic and trivial and you just do it instantly.

williawa 28 Feb 2025 22:06 UTC
1 point
0
in reply to: Lucius Bushnaq’s comment on: Lucius Bushnaq’s Shortform
Well, my model is that the primary reason we’re unable to deal with deceptive alignment or goal misgeneralization is because we’re confused, but that the reason we don’t have a solution to Outer Alignment is because its just cursed and a hard problem.

williawa 28 Feb 2025 22:03 UTC
1 point
2
in reply to: Cole Wyeth’s comment on: My model of what is going on with LLMs
That’s what I also thought haha, else I wouldn’t post it.

williawa 27 Feb 2025 21:01 UTC
3 points
−3
in reply to: Lucius Bushnaq’s comment on: Lucius Bushnaq’s Shortform
Nice, I was going to write more or less exactly this post. I agree with everything in it, and this is the primary reason I’m interested in mechinterp.
Basically “all” the concepts that are relevant to safely building an ASI are fuzzy in the way you described. What the AI “values”, corrigibility, deception, instrumental convergence, the degree to which the AI is doing world-modeling and so on.
If we had a complete science of mechanistic interpretability, I think a lot of the problems would become very easy. “Locate the human flourishing concept in the AIs world model and jack that into the desire circuit. Afterwards, find the deception feature and the power-seeking feature and turn them to zero just to be sure.” (this is an exaggeration)
The only thing I disagree with is the Outer Misalignment paragrpah. Outer Misalignment seems like one of the issues that wouldn’t be solved. Largely due to goodhearts curse type stuff. This article by scott explains my hypothetical remaining worries well https://slatestarcodex.com/2018/09/25/the-tails-coming-apart-as-metaphor-for-life/
Even if we understood the circuitry underlying the “values” of the AI quite well, that doesn’t automatically let us extrapolate the values of the AI super OOD.
Even if we find that, “Yes boss, the human flourishing thing is correctly plugged into the desire thing, its a good LLM sir”, subtle differences in the human flourishing concept could really really fuck us over as the AGI recursively self-improves into an ASI and optimizes the galaxy.
But, if we can use this to make the AI somewhat corrigible, which, idk, might be possible, I’m not 100% sure, maybe we could sidestep some of these issues.
Any thoughts about this?

williawa 23 Feb 2025 21:34 UTC
1 point
0
in reply to: Cole Wyeth’s comment on: My model of what is going on with LLMs
Yeah, that’s my understanding as well. Tell me if your understanding changes further in relevant ways.

williawa 22 Feb 2025 17:02 UTC
7 points
2
in reply to: Cole Wyeth’s comment on: My model of what is going on with LLMs
Yeah, its not just the tokens. It does look at the previous residual streams, What I’m saying is just that each token, the model can only think about internally a fixed amount, bounded by the number of layers. It can NOT think for longer, without writing down its thoughts, as the context grows.
In the article you linked, X is the residual stream, it is a tensor with dimension (length of sequence input) x (dimension of model). But X goes through multiple updates, where each only depends on the previous layer. He is the loop unrolled if L = 2.
So X0 = Embed + PosEmbed
X1 = X0 + MultiheadAttention1(X0)
X2 = X1 + MLP1(X1)
X3 = X2 + MultiheadAttention2(X2)
X4 = X3 + MLP2(X3)
Out = Softmax(Unembed(X4))
The point is that its not like X1[i] = f(X1[:i]). X1 is a function of X_0 only. So the maximum length of any computational path is the HEIGHT of HMYS’ diagram, not the length. You can’t have any computation going from A1 → B1. Only from A1 → B2. Thats what hmys says also
But A1 has direct contributions to B2, C2, D2 and E2 because of attention,
So, unlike in the diagram above. You can’t go immediately to the right, only to the right and up in one computation step.
NOTE: You can also see this just by looking at the code in the document you sent. The for loop is just ran a constant L times. No matter what. What is L? The number of transformer layers. Each innor loop does a fixed amount of computation. And the only thing that changes from time to time, is that there are new tokens written (assuming we’re autoregressively sampling from the transformer in a loop). Ergo, if the model isn’t communicating its “thinking” in writing, it can’t think for longer, as the context grows.

williawa 21 Feb 2025 19:11 UTC
7 points
2
in reply to: Cole Wyeth’s comment on: My model of what is going on with LLMs
I’m not sure this is entirely correct. It’s still true that transformers are bounded in the amount of computation they can do in the residual stream, before the computation has to “cash out” in a predicted token though. In the picture above, its a little unclear, but C2 can only read from A1, B1 and C1, not A2 and B2. There is a maximum length path of computation from one token input to a token output.
The above only establishes that LLM training doesn’t incentivize them to be myopic. Like if you ask a LLM to continue the string “What is 84x53? Answer with” then the next few tokens to predict might be ” one word. Answer:” or something like ” an explanation before you give the final number.”
The above argument just shows that the LLM still might internally be thinking about what 84x53 is on the residual stream of the ” Answer” and the ” with” token, even if that only has relevance for later tokens, and it can easily figure out ” one word”, or ” an explanation before you give the final number.”, without computing the answer.
If you prompt a model with two sentences, they’re probably “thinking” about a bunch of stuff that’s relevant for predicting words many many sentences later.
But they can’t have that complex thoughts unless they write them down. Or, obviously if you just make them bigger they can have more and more complex thoughts, but you’d expect the thoughts they be able to have when they can write stuff, to be a lot more complex than if they have to for example thin deceptive things that don’t appear in writing.
I mean, I don’t want to give Big Labs any ideas, but I suspect the reasoning above implies that the o1/deepseek -style RL procedures might work a lot better if they can think internally for a long time, like the thinking in embedding space model, because gradients from the reward tokens don’t really flow from the placed tokens now. The placed tokens are kind of like the environment in standard RL thinking, but they could actually be differentiated through, turning it more into a standard supervised problem, which is a lot easier than open-ended RL.