Show HN: Truss – Serve any ML model without boilerplate code | Hacker News Reader

Show HN: Truss – Serve any ML model without boilerplate code

github.com

68 pointsphilipkiely9 comments

This looks promising. It feels like for non ML engineers it’s very hard to figure out how to use models as part of vanilla CRUD codebase.

For instance in a Rails app the ML model services would probably be served as a completely external service API generated with something like Truss wrapped in a service class that just exposes the outputs and handles errors/input validation!

In this category I’m a big fan of https://github.com/bentoml/BentoML

What I like about it is their idiomatic developer experience. It reminds me of other Pythonic frameworks like Flask and Django in a good way.

I have no affiliation with them whatsoever, just an admirer.

Big fan of projects like BentoML, the whole ML space is powered by some truly great projects. What we've tried to do with truss is create an experience that is optimizing for simplicity and universality. Our roadmap begins to deviate away from projects like BentoML, and so creating truss was very much motivated by the need to continue this mindset as we march down through that roadmap.

context: I am one of the contributors

Superb product and team.

Worth looking into if you’ve done any engineering work around deploying ML models as a or within a service.

Looks interesting, what if I need to write some logic (pre/post prediction) in the prediction server?

Every time that you make a truss to package your model, you can add custom pre/post processing to every request/response. We've done our best to document it. https://truss.baseten.co/reference/structure#model https://truss.baseten.co/develop/processing

Take a look at `{truss_name}/model/model.py` to add the code.

Please open up an issue or PR if you think we can improve on the documentation!

Looks great! What is the argument to use this over MLFlow model packaging and serving?

Truss contributor here

MLFlow is awesome; it's powerful and flexible and extends through multiple valuable portions of the ML cycle.

Truss aims to be simpler to use in both straightforward and complex cases, requiring fewer lines of code but more importantly coming in later in the modeling process. Truss is opinionated and focused.

But interoperability is important to us and we are looking into ways to connect Truss and MLFlow.

This is likely to share its name with the next Prime Minister of the UK...

Reply on news.ycombinator.com