Hacker News Reader

Comment by BoorishBears | Hacker News Reader

My two cents which you are free to ignore (and I almost implore you to ignore): I'm sure this is useful but as someone who works at an AV company and is building with these new generative AI tools... your psuedo-YC story intro kind of puts me off even wanting to look at the library, because it sets you up as grifters.

The only overlap with what you were doing at Cruise and the problems people building off a REST API wrapper for an LLM are running into are things that all software being pushed into a production environment runs into. High level things like "let's not introduce a regression".

I think if you're talking to investors who don't know better go for it. But if you're posting for technical folks, some of them will be completely put off the moment you try to imply working in MLOps at an AV company makes you any more suited to implement RAG than any suitably experienced engineer who's messed around with embeddings for a month.

View on HN

Thanks for the feedback!

The lesson that we learned at Cruise is that the tough thing when shipping AI software is closing the data loop and integrating all of the steps of the ML lifecycle together, so if you only look at the RAG workflow, I actually agree with you.

The vision for Axilla is that all of the modules interoperate with each other naturally. This means that your production data gets logged such that the datasets can be sent for data processing, labeling, or added to regression test suites. This way, production and development workflows are tied together.

In terms of RAG: how do you test your RAG workflow in an automated way? A lot of people are building these workflows today, but from our customer conversations nearly none of them are testing them or monitoring their performance automatically in production, because most evaluation frameworks don't integrate naturally with document retrieval.

We have a way to go before the framework delivers on its full potential, but we still feel that it's in a useful enough shape for people to use it and contribute today, which is why we open-sourced it while we keep building it.

I just don't see many people shipping AI software right now in the way AI applies to AVs.

But I'm also giving you my two cents as someone who's not only building in the space, but sitting next to * a lot of the people who will use something like this, and there's some real fatigue building around the flood of tooling for "LLMOps".

* figuratively and literally: checking my past events after the sibling mentioned your YC connection and even you and I have been to at least one mutual AI event

—

At the end of the day I get that as a startup you need to weave stories sometimes: If this was a Launch HN I wouldn't have bothered with my comment and that's kind of what my "ignore this" intro is getting at.

But we went from chatbots to selling shovels in a gold rush as the default AI play in the last couple of months. Most builders will take any excuse to assume any given tool is just another rushed shovel. So you don't want to invite the mental friction of "ML vs AI": at most I'd mention you're former coworkers at Cruise and let the people amenable to that connection make it themselves. For the rest of us even knowing two former coworkers are working on something is enough to build some confidence in its staying power.

They are a YC company.

Reply on news.ycombinator.com