Hacker News Reader

Comment by m_ke | Hacker News Reader

Machine learning model management and serving service.

I haven't seen a decent open source framework that makes it easy to package a trained model with prepocessing and postprocessing steps and deploy it behind an API.

Adding performance tracking and model validation on top of that would be great.

Then there are things like queuing/batching and autoscaling.

Closes thing that comes to mind right now is tensorflow serving and their k8s stuff (which it looks like they just renamed to tf-operator and moved to kubeflow org https://github.com/kubeflow/tf-operator)

View on HN

The model development pipeline is still a far cry from the maturity of the software development pipeline, and will have to get there in order to reduce the hands-on heavy lifting required for model development and deployment.

Along with packaging a trained model are things like: Snapshot/versioning of the training and test data used to create the model, versioning of the model, storing versioned models in a model registry, auto-deploying models from the registry to target environments, telemetry from deployed models.

Closest I've found is https://github.com/mitdbg/modeldb, and I've spoken to the woman leading the effort. They still have data versioning as an open question, and don't see the need. But there are training set modification, results RCA, and other use cases that drive the need to catalog training/test data with the model that results.

It'll get there. Just a question of when and how.

This is basically all we do (Disclaimer: This is mainly meant for bigger companies, not startups): https://skymind.ai/platform - reach out if you have any questions. Mail in profile.

One thing I'll say is "just k8s" isn't realistic. You need a lot more than that.

It should run within k8s but should also be minimalistic. Not a lot of platforms provide that. A lot of our docs on the internals are still being built out yet, but we provide everything ranging from an offline install of anaconda to managed connectivity gateway to hadoop and spark clusters.

We also have built in model serving and experiment tracking (which in reality is just a relational database with a rest api automatically integrated in to the platform) - if you're interested in learning more please reach out.

Reply on news.ycombinator.com