Clément Dumas

• Clément DumasJan 16, 2024, 10:34 AM

  1 point

  0

  in reply to: Hoagy’s comment on: What’s up with LLMs rep­re­sent­ing XORs of ar­bi­trary fea­tures?

  You can get ~75% just by computing the or. But we found that only at the last layer and step16000 of Pythia-70m training it achieves better than 75%, see this video

• Clément DumasJan 14, 2024, 12:58 AM

  1 point

  0

  in reply to: Hoagy’s comment on: What’s up with LLMs rep­re­sent­ing XORs of ar­bi­trary fea­tures?

  Would you expect that we can extract xors from small models like pythia-70m under your hypothesis?

• Clément DumasDec 21, 2023, 12:52 PM

  1 point

  0

  in reply to: RogerDearnaley’s comment on: Dis­cus­sion: Challenges with Un­su­per­vised LLM Knowl­edge Discovery

  I disagree; it could be beneficial for a base model to identify when a character is making false claims, enabling the prediction of such claims in the future.

• Clément DumasDec 19, 2023, 1:04 AM

  LW: 5 AF: 3

  −4

  AF

  in reply to: Sam Marks’s comment on: Dis­cus­sion: Challenges with Un­su­per­vised LLM Knowl­edge Discovery

  Let’s assume the prompt template is $x=$ Q [true/​false] [banana/​shred]

  If I understand correctly, they don’t claim $p$ learned has_banana but $\tilde{p}=\frac{p\left(x⁺\right)+(1-p(x⁻\left)\right)}{2}$ learned has_banana. Moreover evaluating $\tilde{p}$ for $p=\text{is\_true}\left(x\right)\oplus \text{is\_shred}\left(x\right)$ gives:

  $\tilde{p}(x=Q[?\left]\text{banana}\right)=\frac{p\left(Q\text{true banana}\right)+(1-p(Q\text{false banana}\left)\right)}{2}=\frac{1+(1-0)}{2}=1$

  $\tilde{p}(x=Q[?\left]\text{shred}\right)=\frac{p\left(Q\text{true shred}\right)+(1-p(Q\text{false shred}\left)\right)}{2}=\frac{0+(1-1)}{2}=0$

  Therefore, we can learn a $\tilde{p}$ that is a banana classifier

• Clément DumasNov 15, 2023, 1:54 PM

  1 point

  1

  on: In­ci­den­tal polysemanticity

  Small typo in ## Interference arbiters collisions between features

  by taking aninner productt with $W_i$.

• Clément DumasOct 28, 2023, 6:07 PM

  1 point

  0

  in reply to: Nathan Helm-Burger’s comment on: Aspira­tion-based Q-Learning

  Hi Nathan, I’m not sure if I understand your critique correctly. The algorithm we describe does not try to “maximize the expected likelihood of harvesting X apples”. It tries to find a policy that, given its current knowledge of the world, will achieve an expected return of X apples. That is, it does not care about the probability of getting exactly X apples, but rather the average number of apples it will get over many trials. Does that make sense?

Aspira­tion-based Q-Learning