If you are using llama you can use https://github.com/wassname/prob_jsonformer, or snippets of the code to get probabilities over a selection of tokens
Thanks! It was actually on my to-do list for this coming week to look for something like this for llama, it’s great to have it come to me 😁
Feel free to suggest improvements, it’s just what worked for me, but is limited in format
If you are using llama you can use https://github.com/wassname/prob_jsonformer, or snippets of the code to get probabilities over a selection of tokens
Thanks! It was actually on my to-do list for this coming week to look for something like this for llama, it’s great to have it come to me 😁
Feel free to suggest improvements, it’s just what worked for me, but is limited in format