As part of the task we are designing, the agent needs access to a LLM API (locally would make the deliverable too big I assume). The easiest would be OpenAI/Anthropic, but that is not static. Alternatively, we could host an LLama in the cloud, but it is not ideal for us to have to keep that running. What is the best way to do this?
Good point! Hmm I think it’s fine to use OpenAI/Anthropic APIs for now. If it becomes an issue we can set up our own Llama or whatever to serve all the tasks that need another model. It should be easy to switch out one model for another.
As part of the task we are designing, the agent needs access to a LLM API (locally would make the deliverable too big I assume). The easiest would be OpenAI/Anthropic, but that is not static. Alternatively, we could host an LLama in the cloud, but it is not ideal for us to have to keep that running. What is the best way to do this?
Good point!
Hmm I think it’s fine to use OpenAI/Anthropic APIs for now. If it becomes an issue we can set up our own Llama or whatever to serve all the tasks that need another model. It should be easy to switch out one model for another.