skull8888888

Totally makes, don't know why I missed agent specific examples... will add them asap.

I focused on the Agent because it was initially built for that internally. But you are right, for the release I made it extremely general and to be used as a foundation for the complex systems.

Noted. I had no intention to be disrespectful in any sense. Just pointing out flaws and reasons why and how current project came to be.

Thank you for the kind words! Feel free to join our discord https://discord.gg/nNFUUDAKub and discuss stuff there. Also ping me there any time, always happy to help you onboard!

Totally agree, I encourage you to give Flow a try! Would be more than happy to help you onboard. Also would love to hear if anything is missing.

Also curious, what kind of agents are you building with LangGraph. Would be more than happy to help you onboard to Flow!

Thank you! Good question. Behind the scenes, state is manage via Semaphore. So each call to .get(...) acquires a semaphore with a single permit so it essentially guarantees thread-safety. Check out how the state is implemented here https://github.com/lmnr-ai/flow/blob/main/src/lmnr_flow/stat...

My main company is Laminar (https://www.lmnr.ai) and we actually help folks ship reliable LLM software to prod. We've seen many of our clients successfully do that. Although, the entire space is very new for sure and things are changing everyday

Thanks! Yep, retry and durability guarantee is on the roadmap for sure, I also wanted to start working on the serverless offering. What kind of workflows are you building with temporal?

thanks for sharing, looks very interesting! Although, it seems to be very high level and hides all the details behind prompt string. I tend to dislike that kind of things. I designed Flow to be as bare bone as possible and give all control to the user

True, graph is not wrong per se, but in my experience I saw a lot of problems, the moment it starts to be a little more dynamic. Main problem is predefined edges. That's where you bump into things like conditional edges, cycles edges and so on. That's also my problem with frameworks like LangGraph. They just get in your way by pushing on things which looked like a good idea in the beginning.

In some sense yes, task dependencies do form a graph. However, what I meant when I was saying that graph is a wrong abstraction, is that predefined edges is a not the best of writing code when dealing with dynamic systems such as AI Agents.

As you said, the easiest conditional workflows are tedious and we have to rely on "conditional" edges to solve the simplest stuff. Then, how would you define cycles or spawning of the same task multiple times as a node-edge relation? It becomes extremely inconvenient and just gets in your way. The reason I built Flow is exactly because I previously relied on a predefined node-edge system which I also built (https://docs.lmnr.ai/pipeline/introduction).

Also it impossible to do MapReduce like operations on node-edge without inventing some special node for handling that stuff.

Idea about comparing LangGraph and Flow is really good and I will work on that asap! What kind of workflows would you love to see implemented in Flow? Btw, there're a lot of examples in the readme and in the /tests folder

You are welcome! The easiest way to get started with BAML on Laminar is with our pipeline builder and Structured Output template. Check out the docs here (https://docs.lmnr.ai/pipeline/introduction)

Thank you! If langtrace sends Otel spans over http or grps we can ingest it! How would one join OTEL GenAI comittee?

Have you tried BAML (https://github.com/boundaryml/baml)? It's really good at structured output parsing. We integrated it directly into our pipeline builder.

Thank you! Yes, our pipeline builder is a way to build LLM pipelines and expose them as API endpoints! All hosted and managed by Laminar.

Hey there, answered similar question here https://news.ycombinator.com/item?id=41453674

We really like langfuse, the team and the product.

Compared to it:

* We send and ingest Otel traces with GenAI semconv

* Provide semantic-event based analytics - you actually can understand what's happening with your LLM app, not just stare at the logs all day.

* Laminar is built be high-performance and reliable from day 0, easily ingesting and processing spikes of 500k+ tokens per seconds

* Much more flexible evals, because you execute everything locally and simply store the results on Laminar

* Go beyond simple prompt management and support Prompt Chain / LLM pipeline management. Extremely useful when you want to host something like Mixture of Agents as a scalable and trackable micro-service.

* It's not released yet, but searchable trace / span data

Thank you! By that we mean, all processing, i.e. data, forward run, evaluator run is done locally in, let's say, Jupyter Notebook. While it's running locally, it sends all the run data / stats to the server for storage and progress tracking.

We love it because we tried putting things into the UI, but found it to be much more limiting rather that letting users design evals and run them however they want.

Hey there, by semantic we mean, embedding text and storing it in the vector DB. Regarding scale, we thought a lot about it, and that's why we process span in the background queue. Tradeoff would be that indexing / embedding would not be real-time as scale will grow. We will also use tiny embedding models, which become better and better.

Hey there, we priced it that way because of our very generous free tier. Your suggestion of usage based pricing also makes sense. So, in our case it might be something: if you pass 50k spans, then you pay something like $0.001 (not a final number) per span. Would you image something like this?

Yes! Our pipeline builder is pretty versatile. You can define conditional routing, parallel branches, and cycles. Right now we support LLM node and util nodes (json extractor). If you can defined your logic purely from those nodes (and in majority of cases you will be), then great, you can host everything on Laminar! You follow this guide (https://docs.lmnr.ai/tutorials/control-flow-with-LLM) it's bit outdated by gives you a good idea on how to create and run pipelines.

Now that you mentioned it, > warnings and errors that mislead you

it really makes sense. I guess what I was pointing into, is that when you have really rich text (in your case it would be error descriptions), searching over them semantically is a must have feature.

But you are right, being an output of LLM is not a requirement.

Hey there, apologies for the late reply.

> Could you clarify why would some need to analyze LLMs out of all the things?

When you want to understand trends of the output of your Agent / RAG on scale, without looking manually at each trace, you need to another LLM to process the output. For instance, you want to understand what is the most common topic discussed with your agent. You can prompt another LLM to extract this info, Laminar will host everything, and turn this data into metrics.

> Why do we need to evaluate LLMs?

You right, devs who want to evaluate output of the LLM apps, truly care about the quality or some other metric. For this kind of cases evals are invaluable. Good example would be, AI drive-through agents or AI voice agents for mortgages (use cases we've seen on Laminar)

Hey there! Good question. Our main distinguishing features are:

* Ingestion of Otel traces

* Semantic events-based analytics

* Semantically searchable traces

* High performance, reliability and efficiency out of the box, thanks to our stack

* High quality FE which is fully open-source

* LLM Pipeline manager, first of it's kind, highly customizable and optimized for performance

* Ability to track progression of locally run evals, combining full flexibility of running code locally without need to manage data infra

* Very generous free tier plan. Our infra is so efficient, that we can accommodate large number of free tier users without scaling it too much.

And many more to come in the coming weeks! On of our biggest next priorities is to focus on high quality docs.

All of these features can be used as standalone products, similar to Supabase. So, devs who prefer keep things lightweight might just use our tracing solution and be very happy with it.

Thank you!

You mean like triggering another processing pipeline from the output of current processing pipeline?

Thank you! We thought a lot about what would make a great title but couldn't really find anything else which would convey info as densely as current title. We also love our current stack :). I think Rust is perfect language to handle span ingestion and it marries perfectly with the rest of our stack.

Valid point. For us searchable and especially semantically searchable, traces / spans really make sense only in the context of tracing LLM apps. And then, we view it as a powerful feature, but, primarily in the context of AI/LLM-native observability platform. For us, the ultimate goal is to build the comprehensive platform, with features which are extremely useful for observability and development of LLM/GenAI apps.

Totally agree, observability is a must for LLM apps. We wanted to build something of extremely high quality but to be affordable for solo devs, that's why open-source and why very generous free tier on our managed version.

Regarding Anthropic instrumentation, we support it out of the box! You don't even need to wrap anything, just do laminar initialize and you should see detailed traces. We also support images! Hit me up at robert@lmnr.ai if you need help onboarding or setting up local version

Turns out there quite a few companies / projects named Laminar. I really like the name, couldn't buy a .com or .ai domain tho, so settled on lmnr.ai. But it's been growing on me.

Hey everybody, today I am launching Laminar AI -> https://www.lmnr.ai (desktop optimized)

TLDR: fast Rust backend and monitoring for AI applications.

Building production-grade AI workflows takes months. Laminar accelerate this process by providing a unified platform for building, deploying and monitoring production-grade AI apps.

Features:

* Flexible pipeline builder UI for LLM workflows

* All pipeline workflows are executed on fastest Rust backend and exposed as API endpoints

* Chain-of-thought agent and semantic search infrastructure are first class citizens

* Parallel workflows for different models

* Parallel executions with different inputs

* Log inputs, outputs and metadata for every action node in pipeline graph

* Simple and powerful workflow version control with an ability to deploy to production in one click

Soon:

* evaluations i.e. unit tests for your workflows

* fine-tuning models based on your production log data

* other models beyond LLM: STT/TTS, OCR, vision

If you are interested, read further description and consider upvoting it on Product Hunt -> https://www.producthunt.com/posts/laminar-2