brunaxLorax

Cool

It's hard to keep track of all those launches, I agree. I would separate 2 usages of LLMs: - LLM is integrated into a workflow/app for a classification or whatever job: scoring outputs (evals, KPIs) and choosing a fit between quality and cost is enough - LLM is used as a tool you interact with (like coding or research): here is where you have to know your brushes. As humans we get used very quickly by the model personality, the format of output, the way it "thinks" and interacts.

a French human doing its best to sound punchy in English :)

Yes. orchestrator > smart router

Sounds like an orchestrator for me ratter than a router

Agree

We tried different things: heuristic, TF-IDF and LLMs. I voluntarily didn't talk about the tech because it doesn't fix the problems mentioned by switching models on the fly.

100% agree. MoE is the perfect example: reducing FLOPs while keeping intelligence

All inference providers (labs and neoclouds like TogetherAI or Fireworks) are incentivized to be efficient to be more competitive. For example MoE reduces compute without reducing output quality. I would not be surprised if they end up implementing some kind of internal routing at some point.

Cheers, good luck!

Could be. In that case many routers are concerned as they all have more or less the same category buckets.

Yes, providers have downtime and retry/fallback solves this problem.

I didn't know that! I thought that all models had the same features no matter the provider

I didn't know about Tokenless, the approach seems really innovative, if it works it fixes the "Complexity cannot be deduced from the prompt alone" problem. However you still have the other hidden costs: cache, breaking behavior consistency, and unpredictability.

Your point is interesting, you say that task specificity is easier to classify than task complexity, which I agree - I didn't mention it but we had task specificity routing too :). My opinion is that in many cases task specificity calls are easy to distinguish at build time, and therefore you can isolate them and attach the right model/settings beforehand, so you have less need for a smart routing on the fly.

If your router has GDPR/ZDR/EU compliant, it doesn't make your provider compliant. You're just adding a middleman.

Agree. The orchestrator pattern (big model managing smaller ones) works already well

I agree. However as you said, you can consider code agents like human contributors. My feedback is that an army of Claude Code instances with a strict CLAUDE.md file is more rigorous than a human team.

I recently came across https://claudecode.run/ (not affiliated). There are currently several similar products that help that multi-instance Claude Code layout

How quickly? There is no limit to that. If I can do my day of work in few hours I would be glad to take the afternoon off or to go twice as fast (importunately that's more the second!)

With git worktrees you can you have your agents working on literally different folders so they don't step up on each other.

Fixed ! thank you

Hello, Manifest dev here, most backend-as-a-services rely on an UI and it's harder for AIs. In comparison, Manifest high-level DSL is really understandable by gen AIs. It's also very easy to validate in comparison to bootstrap frameworks that come with a lot of files.

But you are right we should explain it somewhere !

correct :)

Hello ! Manifest dev here, they are on the way ! There is a DB sync at the moment that does the job but we are going to implement the migrations soon :)

It will be an express based REST API that connects to an external Database.

The API will have to expose endpoints and do CRUD operations for the most part.

(dev here) Backend-as-a-service products are very popular right now because they remove or simplify a lot of the backend complexity. It comes super handy for some projects.

Yes, the DB is SQLite. We chose it among others as it is file based and thus you get get up and running in seconds.

Authentication and authorization are key features, I agree. They were not integrated in the POC but they will come very soon. There actually is auth for the admin panel, I just need to standardize it for other entities.

I got it for the CMS use case, Manifest's aim is not to be a competitor of large frameworks that gives you the control of everything. We rather think that it will fit for another typology of projects. You can use it as a headless CMS, but there already is products like Strapi or Directus that get the job done. I am thinking more about projects with more "app" logic, but the next step is adding custom logic to it.

Migrations are a tricky point I agree. In your system, do you keep your original model files as they were at the beginning or do you change them too ?

I am thinking about having some hooks for the custom logic. I like the entity listeners/subscribers from typeORM: https://orkhan.gitbook.io/typeorm/docs/listeners-and-subscri... but I the end I guess we would probably need some functions stored somewhere. Not sure how to implement that though

Yes there is a lot of similarities in the foundation and the syntax. The end goal is different though as Manifest aims to be a production-ready product: it already comes with the admin panel and will soon include features like authentication, authorization and so on.

Mmmm... It comes from ORMs in general I guess. I appreciate the effort of some ORMs to get a nice syntax, making it feel like natural language. I guess Laravel Eloquent was my first crush in my PHP days. Then modern ORMs like TypeORM and Prisma do an excellent job IMO.

I also like the idea of transposing ORM-style queries in the browser to abstract the whole API response-request part.