Jakub Smékal

Ex­cur­sions into Sparse Au­toen­coders: What is monose­man­tic­ity?

• Jakub SmékalFeb 20, 2024, 5:43 PM

  1 point

  0

  in reply to: Joseph Bloom’s comment on: Open Source Sparse Au­toen­coders for all Resi­d­ual Stream Lay­ers of GPT2-Small

  Neel was advised by the authors that it was important minimise batches having tokens from the same prompt. This approach leads to a buffer having activations from many different prompts fairly quickly.

  Oh I see, it’s a constraint on the tokens from the vocabulary rather than the prompts. Does the buffer ever reuse prompts or does it always use new ones?

• Jakub SmékalFeb 19, 2024, 7:24 PM

  3 points

  0

  on: Open Source Sparse Au­toen­coders for all Resi­d­ual Stream Lay­ers of GPT2-Small

  • We store activations in a buffer of ~500k tokens which is refilled and shuffled whenever 50% of the tokens are used (ie: Neel’s approach).

  I am not sure I understand the reasoning around this approach. Why do you want to refill and shuffle tokens whenever 50% of the tokens are used? Is this just tokens in the training set or also the test set? In Neel’s code I didn’t see a train/​test split, isn’t that important? Also, can you track the number of epochs of training when using this buffer method (it seems like that makes it more difficult)?

Com­mu­ni­ca­tion, con­scious­ness, and be­lief strength measures

Mea­sur­ing pre-peer-re­view epistemic status

• Jakub SmékalJan 25, 2024, 5:19 AM

  2 points

  0

  on: Sparse Au­toen­coders Work on At­ten­tion Layer Outputs

  Hey, great post! Are your code or autoencoder weights available somewhere?

Start­ing in mechanis­tic interpretability

Carv­ing up prob­lems at their joints