skunnavakkam

• skunnavakkamApr 2, 2025, 7:19 AM

  1 point

  0

  in reply to: zookini’s comment on: Re­dun­dant At­ten­tion Heads in Large Lan­guage Models For In Con­text Learning

  norm \in \mathbf{R}, doesn’t matter

• skunnavakkamDec 4, 2024, 10:35 PM

  4 points

  0

  on: Ex­plore More: A Bag of Tricks to Keep Your Life on the Rails

  I’ve found the part about applying random search to be the among the best takeaways I had from PAIR! Novelty for the sake of Novelty is not a terrible idea. Specifically, I’ve found that even if you don’t like the things you do, it makes it much easier to then make progress towards the larger goal

Re­dun­dant At­ten­tion Heads in Large Lan­guage Models For In Con­text Learning