I’m unsure whether it’s a good thing that LLaMA exists in the first place, but given that it does, it’s probably better that it leak than that it remain private.
What are the possible bad consequences of inventing LLaMA-level LLMs? I can think of three. However, #1 and #2 are of a peculiar kind where the downsides are actually mitigated rather than worsened by greater proliferation. I don’t think #3 is a big concern at the moment, but this may change as LLM capabilities improve (and please correct me if I’m wrong in my impression of current capabilities).
Economic disruption: LLMs may lead to unemployment because it’s cheaper to use one than to hire a human to do the same work. However, given that they already exist, it’s only a question of whether the economic gains accrue to a few large corporations or to a wider mass of people. If you think economic inequality is bad (whether per se or due to its consequences), then you’ll think the LLaMA leak is a good thing.
Informational chaos: You can never know whether product reviews, political opinions, etc. are actually genuine expressions of what some human being thinks rather than AI-generated fluff created by actors with an interest in deceiving you. This was already a problem (i.e. paid shills), but with LLMs it’s much easier to generate disinformation at scale. However, this problem “solves itself” once LLMs are so easily accessible that everyone knows not to trust anything they read anyway. (By contrast, if LLMs are kept private, AI-generated content seems more trustworthy because it comes in a wider context where most content is still human-authored.)
Infohazard production: If e.g. there’s some way of building a devastating bioweapon using household materials, then it’d be really bad if LLaMA made this knowledge more accessible, or could discover it anew. However, I haven’t seen any evidence that LLaMA is capable of discovering new scientific knowledge that’s not in the training set, or that querying it to surface existing such knowledge is any more effective than using a regular search engine. But this may change with more advanced models.
I want to say something like: “The bigger N is, the bigger a computer needs to be in order to implement that prior; and given that your brain is the size that it is, it can’t possibly be setting N=3↑↑↑↑↑3.”
Now, this isn’t strictly correct, since the Solomonoff prior is uncomputable regardless of the computer’s size, etc. - but is there some kernel of truth there? Like, is there a way of approximating the Solomonoff prior efficiently, which becomes less efficient the larger N gets?