Ask HN: Custom AI LLM Solutions vs. Open Source Tools?

2 points·meiraleal··5 comments

Has anybody else chosen the route of developing a fully customized solution to integrate LLMs into their workflow instead of using some of the available options (AutoGPT, AgentGPT, LangChain, and others)?

I'd love to hear about your experiences and any challenges you faced. So far, my most challenging issue has been creating an output format that doesn't fail 20-30% of the time.

View on HN

I have found working with DSPy to be a nice middle ground. The python script contains functions that call different DSPy optimizers (the optimized prompt packaged as a function; tested and iterated individually in a Jupyter notebook). Using DSPy `TypedPredictors` returns structured output that I then parse (ie. with conditions/loops) within the larger python script.

Can't say for sure yet if the project will work, but this workflow seems OK because DSPy abstracts away the CoT optimization with still giving me full control of the inputs/outputs in a python multi-agent script

DSPy looks interesting! But ironically I feel like learning a bit of what it does and implement some parts by my own. It is so fast to code something usable with Claude 3 opus.

I’m developing my own mainly because most things are in python and I prefer JS

That's one of my issues but nowadays there are good options written in JavaScript too and a langchain port.

Make your project open source so we stop starting from scratch :)

Reply on news.ycombinator.com