Vikrant Varma

JumpReLU SAEs + Early Ac­cess to Gemma 2 SAEs

Im­prov­ing Dic­tionary Learn­ing with Gated Sparse Autoencoders

[Full Post] Progress Up­date #1 from the GDM Mech In­terp Team

[Sum­mary] Progress Up­date #1 from the GDM Mech In­terp Team

Dis­cus­sion: Challenges with Un­su­per­vised LLM Knowl­edge Discovery

Ex­plain­ing grokking through cir­cuit efficiency

• Vikrant Varma 28 Nov 2022 16:18 UTC

  LW: 3 AF: 1

  0

  AF

  in reply to: Ramana Kumar’s comment on: Mechanis­tic anomaly de­tec­tion and ELK

  To add some more concrete counter-examples:

  • deceptive reasoning is causally upstream of train output variance (e.g. because the model has read ARC’s post on anomaly detection), so is included in π.

  • alien philosophy explains train output variance; unfortunately it also has a notion of object permanence we wouldn’t agree with, which the (AGI) robber exploits

Refin­ing the Sharp Left Turn threat model, part 2: ap­ply­ing al­ign­ment techniques

Threat Model Liter­a­ture Review

Clar­ify­ing AI X-risk

More ex­am­ples of goal misgeneralization

Refin­ing the Sharp Left Turn threat model, part 1: claims and mechanisms

• Vikrant Varma 10 May 2022 16:36 UTC

  1 point

  AF

  on: Knowl­edge is not just mu­tual information

  Thanks for this sequence!

  I don’t understand why the computer case is a counterexample for mutual information, doesn’t it depend on your priors (which don’t know anything about the other background noise interacting with photons)?

  Taking the example of a one-time pad, given two random bit strings A and B, if C = A ⊕ B, learning C doesn’t tell you anything about A unless you already have some information about B. So I(C; A) = 0 when B is uniform and independent of A.

  Over time, the photons bouncing off the object being sought and striking other objects will leave an imprint in every one of those objects that will have high mutual information with the position of the object being sought.

  If our prior was very certain about any factors that could interact with photons, then indeed the resulting imprints would have high mutual information, but it seems like you can rescue mutual information here by saying that our prior is uncertain about these other factors so the resulting imprints are noisy as well.

  On the other hand, it seems correct that an entity that did have a more certain prior over interacting factors would see photon imprints as accumulating knowledge (for example photographic film).

ELK con­test sub­mis­sion: route un­der­stand­ing through the hu­man ontology