The TL;DR is that a while back, someone figured out that giving humans a low-dose horse tranquilizer cured depression (temporarily).
I don’t know (and I don’t want to know) how they figured that out, because the story in my head is funnier than anything real life could come up with.
Well, I mean, it’s also a human tranquilizer. I worry that calling medications “animal-medications” delegitimize their human use-cases.
I think part of the reason why these odds might seem more off than usual is that Ether and other cryptocurrencies have been going up recently which means there is high demand for leveraged positions. This in turn means that crypto lending services such as aave having been giving ~10% APY on stablecoins which might be more appealing than a riskier, but only a bit higher, return from prediction markets.
Are you sure you would need to fine-tune Llama-3? It seems like there are many reports that using a refusal steering vector/ablation practically eliminates refusal on harmful prompts, perhaps that would be sufficient here?
Do labs actually make any money on these subscriptions? It seems like the average user is using far more than 20$ of requests (going by the price for API requests which surely can’t have a massive margin?).
Obviously they must gain something or they wouldn’t do it, but it seems likely the benefits are more intangible, gaining market share, generating hype and attracting API users etc. These benefits seem like they may arise from free usage as well.
Wasn’t the surprising thing about GPT-4 that scaling laws did hold? Before this many people expected scaling laws to stop before such a high level of capabilities. It doesn’t seem that crazy to think that a few more OOMs could be enough for greater than human intelligence. I’m not sure that many people predicted that we would have much faster than scaling law progress (at least until ~human intelligence AI can speed up research)? I think scaling laws are the extreme rate of progress which many people with short timelines worry about.
It also seems likely that the Nano models are extremely overtrained compared to the scaling laws. The scaling laws are for optimal compute during training, but here they want to minimize inference cost so it would make sense to train for significantly longer.
It’s interesting that it still always seems to give the “I’m an AI” disclaimer, I guess this part is not included in your refusal vector? Have you tried creating a disclaimer vector?
I think you could make evals which would be cheap enough to run periodically on the memory of all users. It would probably detect some of the harmful behaviors but likely not all of them.
We used memory partly as a proxy for what information a LLM could gather about a user during very long conversation contexts. Running evals on these very long contexts could potentially get expensive, although it would probably still be small in relation to the cost of having the conversation in the first place.
Running evals with the memory or with conversation contexts is quite similar to using our vetos at runtime which we show doesn’t block all harmful behavior in all the environments.