Whiffed attempt for me. Writing this as the last embers of too-much-coffee fade away, so it may not be coherent.
I tried some of the existing bots, and last minute I concluded was actually a LOT of low hanging fruit and maybe I could have an impact. So I frantically tried to pull something together all day Friday, and now into Saturday morning—couldn’t pull it together. Crashed and burned on some silly Windows problems, eventually bit the bullet and installed WSL/conda/all that, drank a second night pot of coffee… and then finally the treaure at the end of the rainbow, langchain. I’ve been hardcoding raw python prompt chains all this time. This contest was my excuse to finally bit the bullet and modernize my outdated and inefficient LLM workflow. I bet I’ll be kicking myself for not using this modern tool!
And I was was utterly flummoxed by langchain, to be honest. I’m not a great programmer but I spend tons and tons of time experimenting and playing with prompt chaining and all that langchainy style stuff. I just code it all in a single raw python scripts full of horrible regexes and too many IF statements, like a caveman. And yeah, the langchain vector database worked out the box, first try. If the hard things are this easy in langchain, then surely it’s all smooth sailing from here! And then I sat down to dig in and do the ‘low hanging fruit work’ (experiment and iterate on different chains, workflows, find good metrics, optimize token allocation in context windows, the nuts and bolts of LLM interactions. And I was just baffled, it felt like I was working blind.
I mean, I did see langchain had a ‘tracer’ tool. I know it came out recently and it’s still in private waitlist access. So given that context I just assumed that obviously tracer isn’t like a core requirement. It’s got to be a fancy ui visualization frosting on top of boring log files or some other system. That’s classically how an open source company makes money. Surely tracer can’t be the only way to easily see everything? Tracer just came out, it’s absurd to think it’s the only way to see this stuff. I mean, how were people even using langchain at all before trace? Isn’t this like the most basic day 1 function when you work with LLMs? And honesty I still don’t know if I AM missing something obvious, at the end of the night, 4:30 AM EST.
I was able to get some outputs printed to shell, hooking functions, but then I changed something and had to do it again. Finally (HOURS LATER) I bit the bullet, double check the tracer webpage and saw the waist and also DOCKER install. That still seemed excessive, I didn’t even have Docker installed, but whatever. Tracer worked fine, I kicked myself for waiting so long, and I still had a couple hours. Enough for low hanging fruit… maybe. But I’m still being moderately flummoxed by stuff I assumed would be trivial in langchain. Like for example, a lot of the parts of langchain measure length in raw characters instead of tokens. I just assumed I was missing something obvious again. Is there a reason I should care about the character count instead of tokens? Maybe for a third party website? Maybe langchain has automated token management intelligently, and I’m overthinking this? Like here I am going ‘Okay so these documents here written in this writing style, I guess I can estimate the tokens from the character count to get an upper bound and hope for the best’ and this… this can not be the way.
Just ranting as I crash. If I could go back in time and just tell myself “just install the tracer” that alone might have salvaged it. I can not believe I got hung up so long just trying to see what exactly the OpenAI server was getting and receiving.
Whiffed attempt for me. Writing this as the last embers of too-much-coffee fade away, so it may not be coherent.
I tried some of the existing bots, and last minute I concluded was actually a LOT of low hanging fruit and maybe I could have an impact. So I frantically tried to pull something together all day Friday, and now into Saturday morning—couldn’t pull it together. Crashed and burned on some silly Windows problems, eventually bit the bullet and installed WSL/conda/all that, drank a second night pot of coffee… and then finally the treaure at the end of the rainbow, langchain. I’ve been hardcoding raw python prompt chains all this time. This contest was my excuse to finally bit the bullet and modernize my outdated and inefficient LLM workflow. I bet I’ll be kicking myself for not using this modern tool!
And I was was utterly flummoxed by langchain, to be honest. I’m not a great programmer but I spend tons and tons of time experimenting and playing with prompt chaining and all that langchainy style stuff. I just code it all in a single raw python scripts full of horrible regexes and too many IF statements, like a caveman. And yeah, the langchain vector database worked out the box, first try. If the hard things are this easy in langchain, then surely it’s all smooth sailing from here! And then I sat down to dig in and do the ‘low hanging fruit work’ (experiment and iterate on different chains, workflows, find good metrics, optimize token allocation in context windows, the nuts and bolts of LLM interactions. And I was just baffled, it felt like I was working blind.
I mean, I did see langchain had a ‘tracer’ tool. I know it came out recently and it’s still in private waitlist access. So given that context I just assumed that obviously tracer isn’t like a core requirement. It’s got to be a fancy ui visualization frosting on top of boring log files or some other system. That’s classically how an open source company makes money. Surely tracer can’t be the only way to easily see everything? Tracer just came out, it’s absurd to think it’s the only way to see this stuff. I mean, how were people even using langchain at all before trace? Isn’t this like the most basic day 1 function when you work with LLMs? And honesty I still don’t know if I AM missing something obvious, at the end of the night, 4:30 AM EST.
I was able to get some outputs printed to shell, hooking functions, but then I changed something and had to do it again. Finally (HOURS LATER) I bit the bullet, double check the tracer webpage and saw the waist and also DOCKER install. That still seemed excessive, I didn’t even have Docker installed, but whatever. Tracer worked fine, I kicked myself for waiting so long, and I still had a couple hours. Enough for low hanging fruit… maybe. But I’m still being moderately flummoxed by stuff I assumed would be trivial in langchain. Like for example, a lot of the parts of langchain measure length in raw characters instead of tokens. I just assumed I was missing something obvious again. Is there a reason I should care about the character count instead of tokens? Maybe for a third party website? Maybe langchain has automated token management intelligently, and I’m overthinking this? Like here I am going ‘Okay so these documents here written in this writing style, I guess I can estimate the tokens from the character count to get an upper bound and hope for the best’ and this… this can not be the way.
Just ranting as I crash. If I could go back in time and just tell myself “just install the tracer” that alone might have salvaged it. I can not believe I got hung up so long just trying to see what exactly the OpenAI server was getting and receiving.