Thank you for the kind words!
31 karma · joined December 17, 2022
Apologies if you wasted time trying it in other regions!
So I got this idea - what if we can just write function / class definitions and let AI generate the code for us. One thing led to another and after a fun weekend I got something working.
It’s a simple python decorator that uses a function definition to generate the function code. It works for async functions and class methods, and it caches the generated code locally. There are some fun use cases like AI-generated fastAPI endpoints and agent tools.
I also realized that it somewhat incentivizes test-driven development because if you write the tests ahead of time you don’t only make sure that the generated function code works but it is generated and cached before the actual program runs.
I haven’t had enough time to do a proper comparison but I wonder if this test-driven approach produces better code vs traditional vibe-coding.
I would love to hear your thoughts. Cheers!
The main difference between langchain and LLMFlows is in the philosophy - langchain has a "chain for everything" approach where chains come with multiple LLM calls, opinionated internal logic, and built-in default prompts. On the other hand llmflows has a "simple, explicit, transparent" approach where the goal is to allow developers to easily build their own chains (or flows) and have full control and transparency over the apps.
If you are curious check out the intro section of this blog post I made some time ago: https://llmflows.substack.com/p/introduction-to-llmflows
You might find this guide useful to your use case:
https://llmflows.readthedocs.io/en/latest/user_guide/Async%2...
It shows how you can connect multiple similar flowsteps to a previous flowstep and they will all run in parallel.
Building apps with LLMs is fundamentally an exercise of manipulating strings and making API calls (Both LLM and vector db APIs). When apps are more complex and start including multiple LLM calls and prompts that can change dynamically some additional challenges start to emerge. e.g figuring out:
- When was a particular LLM called?
- How much time did it take?
- What were the input variables?
- What did the prompt look like?
- What was the exact configuration of each LLM?
- How many times did we retry the request?
- What was the raw data the API returned?
- What was exactly returned from the vector store and fed into an LLM?
- How many tokens were used?
- What was the final result for each call?
- How do you make API calls in parallel?
I think a framework like this should provide abstractions that allow people to focus on the important part like prompt engineering and productionizing the app and worry less about how to figure out stuff like this.
Right now LLMFlows supports only OpenAI models and Pinecone but I am working on classes for Chroma and Weaviate. And I would like to also provide support for Bard and Claude once I get access.