LLM App Stack – a.k.a. Emerging Architectures for LLM Applications
github.com
github.com
I accidentally stepped into seemingly endless demand for AI projects this year, and it has been a god-send for my crushing burnout. Truly interesting stuff that makes me feel like I felt when I was a kid learning unix.
I can accomplish such objectively cool things that add measurable value with these new tools, but the hype makes it all feel mildly gross.
I am both excited and feel the need to wash my hands.
If you need any help, let me know!
I'm a freelancer and have been doing various LLM projects too, using RAGs (Zep, Bedrock KB), promptfoo (prompt testing).
A lot of it is deployed via AWS CDK, using serverless components, such as Step Functions - great for orchestration.
Using TypeScript end to end.
We built a lead qualification and sales coaching plug-in for a CRM, the thing has a lot of personality. IMO it was a project conceived to automate sales that ended up automating sales coaching.
From there I've done a fair bit of red teaming customer-facing chatbots, then helped save some stuck projects because these approaches to problem solving are so new.
Something that was really satisfying was a project to introduce natural language queries to a ridiculous array of internal corporate databases without sending private data out of the network. That thing is like an extremely boring corporate Jarvis.
I have absolutely zero traditional "AI" skill set, but I guess there's a lot of room downstream for implementation engineering.
It's actually a really fun problem so long as you're burning up tokens on someone else's api key