Is AI back end complexity a pain point? Seeking feedback on an open source idea | Hacker News Reader

Is AI back end complexity a pain point? Seeking feedback on an open source idea

github.com

2 pointsliurenju4 comments

Hi all,

I am a founder working on AI applications, and I have noticed that building the backend for AI apps feels much more complex and fragmented than for traditional SaaS. Things like usage-based billing, managing credits, LLM streaming (with session resuming), user behavior analytics, and integrations with multiple model providers all add a lot of overhead before you can even focus on the product itself.

I am thinking of starting an open source project called AiBase (https://github.com/liurenju/AiBase) to handle these backend pain points out of the box, so teams can focus on building their core AI features instead of wrestling with infrastructure.

For those building or planning to build AI products, do you feel these are major pain points? Would you use an open source Backend as a Service for this, or do you prefer rolling your own solution? What would you want to see in such a project for it to actually be useful? Would love to hear your experiences and honest opinions, including “this is not a real problem,” “I would never use BaaS for AI,” or any similar feedback.

> Things like usage-based billing, managing credits, LLM streaming (with session resuming), user behavior analytics, and integrations with multiple model providers all add a lot of overhead before you can even focus on the product itself.

That doesn't make sense to me. The product needs should dictate whether or not you need those things. You shouldn't be boiler plating them in without a product-driven reason. So if you found the need and are responding to that need... you are focusing on the product.

Thanks for your comments. We distill these features as our priority because our current product SellToSky needs them and we spent significant amount of time building it. We'll not re-invent the wheel but rather glue the available open source project + any missing parts. The whole purpose is to abstract the backend needs as a service -> reducing the repetitive works from the fellow AI developers/founders.

That's why I am coming here to see whether any other developers or founders experienced the same pain?

The exact same set of pains, that could be solved by a single solution? Unlikely.

You'd need to have more of a menu of services available, likely with variations of each. It might be feasible, especially as more people try to launch AI-driven apps without actually knowing what they are doing. But it is going to be a fine line between a back-end platform vs. just doing consulting gigs with potential customers.

You would likely end up catering to people who are less experienced, as no individual piece is all that difficult, so experienced coders would just build it themselves. In particular if they are building a product whose primary transaction is AI-driven, conventional wisdom is to avoid buying 3rd party solutions for your core business value.

Reply on news.ycombinator.com