Neel Nanda comments on Ophiology (or, how the Mamba architecture works)

[Neel Nanda]

Thanks for the clear explanation, Mamba is more cursed and less Transformer like than I realised! And thanks for creating and open sourcing Mamba Lens, it looks like a very useful tool for anyone wanting to build on this stuff