If everyone affirms this is indeed all the major arguments for open weights, then I can at some point soon produce a polished full version as a post and refer back to it, and consider the matter closed until someone comes up with new arguments.
Feels like the vast majority of the benifits Zuck touted could be achieved with: 1. A cheap, permissive API that allows finetuning, some other stuff. If Meta really want people to be able to do things cheaply, presumably they can offer it far far cheaper than almost anyone could do it themself without directly losing money. 2. A few partnerships with research groups to study it, since not many people have enough resources that doing research on a 405B model is optimal, and don’t already have their own. 3. A basic pledge (that is actually followed) to not delete everyone’s data, finetunes, etc. to deal with concerns about “ownership”
I assume there are other (sometimes NSFW) benefits he doesn’t want to mention, because the reason the above options don’t allow those activities is that Meta loses reputation from being associated with them even if they’re not actually harmful.
Are there actually a hundred groups who might usefully study a 405B-parameter model, so Meta couldn’t efficiently partner with all of them? Maybe with GPUs getting cheaper there will be a few projects on it in the next MATS stream? I kinda suspect that the research groups who get the most out of it will actually be the interpretability/alignment teams at Google and Anthropic, since they have the resources to run big experiments on Llama to compare to Gemini/Claude!
A basic pledge (that is actually followed) to not delete everyone’s data, finetunes, etc.
Such a commitment would not be credible. Even if you’re confident Meta won’t do the classic Google thing of sunsetting the service, or go broke, or change CEOs, they’d still be subject to things like DMCA takedowns that running locally safeguards you from.
This is a stronger argument than I first thought it was. You’re right, and I think I have underestimated the utility of genuine ownership of tools like fine-tunes in general.
I would imagine it goes api < cloud hosting < local hosting in terms of this stuff, and with regular local backups (what’s a few TB here or there) then it would be feasible to protect your cloud hosted 405B system from most takedowns as long as you can find a new cloud provider. I’m under the impression that the vast majority of 405B users will be using cloud providers, is that correct?
I think a lot of it is simply just eating away at the margins of companies and product that might become larger in the future. Even if they are not direct competitors, it’s still tech investment money going away from their VR bets into AI. Also big companies fully controlling important tech products has proven to be a nuisance to Meta in the past.
Feels like the vast majority of the benifits Zuck touted could be achieved with:
1. A cheap, permissive API that allows finetuning, some other stuff. If Meta really want people to be able to do things cheaply, presumably they can offer it far far cheaper than almost anyone could do it themself without directly losing money.
2. A few partnerships with research groups to study it, since not many people have enough resources that doing research on a 405B model is optimal, and don’t already have their own.
3. A basic pledge (that is actually followed) to not delete everyone’s data, finetunes, etc. to deal with concerns about “ownership”
I assume there are other (sometimes NSFW) benefits he doesn’t want to mention, because the reason the above options don’t allow those activities is that Meta loses reputation from being associated with them even if they’re not actually harmful.
Are there actually a hundred groups who might usefully study a 405B-parameter model, so Meta couldn’t efficiently partner with all of them? Maybe with GPUs getting cheaper there will be a few projects on it in the next MATS stream? I kinda suspect that the research groups who get the most out of it will actually be the interpretability/alignment teams at Google and Anthropic, since they have the resources to run big experiments on Llama to compare to Gemini/Claude!
Such a commitment would not be credible. Even if you’re confident Meta won’t do the classic Google thing of sunsetting the service, or go broke, or change CEOs, they’d still be subject to things like DMCA takedowns that running locally safeguards you from.
This is a stronger argument than I first thought it was. You’re right, and I think I have underestimated the utility of genuine ownership of tools like fine-tunes in general. I would imagine it goes api < cloud hosting < local hosting in terms of this stuff, and with regular local backups (what’s a few TB here or there) then it would be feasible to protect your cloud hosted 405B system from most takedowns as long as you can find a new cloud provider. I’m under the impression that the vast majority of 405B users will be using cloud providers, is that correct?
I think a lot of it is simply just eating away at the margins of companies and product that might become larger in the future. Even if they are not direct competitors, it’s still tech investment money going away from their VR bets into AI. Also big companies fully controlling important tech products has proven to be a nuisance to Meta in the past.