Used to kick models in the head to see how they ticked at Google Brain and OpenAI. Now Co-Founder at dmodel.ai, a private mechinterp lab.
Outside of AI, my background is in formal methods and in high energy theory.
Anish Tondwalkar
Karma: 7
How Language Models Understand Nullability
Just about every web site I visit, I’ve discovered, has at least a couple of third-party dependencies, whose provenance I don’t trust, and which I’d rather not spend (even a minuscule proportion of) my energy bill on.
A much bigger win the the same vein is your mobile data bill! I’ve been using Brave on Android with JS off by default and adblocking and my month-to-month data usage has fallen precipitously. (One can also use Firefox mobile with various add-ons to achieve the same effect, obviously)
Thanks for writing this and for the diagram, which I think is clearer than anything I’ve seen that’s attempted to clarify this mess.
That said, the terminology remains fantastically confusing.
> Mistral’s models are also not open source, but in a slightly more nuanced manner. Instead of releasing all artifacts describing their models, Mistral licensed the model weights using the Apache 2.0 license, which meets the requirements for a license to be Open Source. Unfortunately, however, no other artifacts were released. As a result, Mistral’s models can be used as-is by anyone, but the transparency that should go hand-in-hand with Open Source is no longer present.
IIUC, this means that Mistral’s models are both Open Source (in the sense of the license being an Open Source license), and Open Source AI (in the sense that they adhere to the OSAID—the first question in the OSAID FAQ is “Why is the original training dataset not required?”), but it is not classified as Open Source according to the classification scheme used in the diagram. Perhaps we need a 5th term to make these clearer?