Show HN: Convect – Instant Serverless Deployment of ML Models
convect.ml
convect.ml
Convect deploys machine learning (ML) models to instantly callable, serverless API endpoints. Using Convect, Jupyter notebook users can deploy trained models from their notebooks and share them with the world in seconds. No web development or infrastructure experience is needed. Convect simplifies the process by being more opinionated than other model deployment workflows.
To give Convect a try, visit https://app.convect.ml. You can also try out models without signing into an account on the demo page (https://app.convect.ml/#/demo). I would love your feedback.
Some background/context:
Deploying ML models to be used in production entails a different set of skills than training models in a sandbox environment and can get pretty complicated depending on what you’re trying to do. For many data scientists, this “sandbox” environment is a Jupyter notebook. One common approach to “deploying to production” that I saw while previously working as a data scientist at Intuit, is turning a model trained with scikit-learn in a notebook into an API endpoint.
From my experience, there are a few ways to deploy a model to an API endpoint and all of them involve a nontrivial level of effort and time. Examples of some of the steps in the process include pickling a model and uploading it to cloud storage, Dockerizing a model’s prediction code and environment, deploying a Flask app, or getting set up with an ML framework (e.g. MLFlow) or platform (e.g. SageMaker) so you can use the deployment feature in their SDK.
While complex workflows make sense for deploying complex models, I haven’t seen any dead simple deployment solutions for simple models, and that’s what I am working on building with Convect. In this case, simplicity comes at the cost of flexibility, i.e. you give up the ability to customize your infrastructure and runtime environment in exchange for a simple, one-click workflow. My hypothesis is that this tradeoff is worth it in many situations, and I’m curious to see what people are enabled to build when this aspect of the ML workflow is drastically simplified.
Under the hood, Convect creates two artifacts by serializing 1) the model prediction code and 2) all the variables that are in scope in the Python session at deployment time. I’ve made this part of the deployment code public here: https://github.com/convect-ml/convect/. These artifacts are then loaded and executed in an AWS Lambda function upon invocation by an API Gateway endpoint.
I’ve talked with 50+ data scientists at small to medium size companies (2-300 employees) and many have identified deployment as a pain point in their workflows. I’ve also spoken with a few data scientists who have indicated that this would help save time on their after-work/weekend side projects.
I’m sharing this now on Show HN because I’d love for people to try out Convect and to hear about how you use it or how I can improve it to make it useful. I’ve also put together a gallery of examples for training and deploying models to make it easy to quickly get started (https://convect.readme.io/docs) and provided example endpoints that you can use query or even build ML-powered apps on top of (https://app.convect.ml/#/demo). Find out more at https://convect.ml. Thanks for having a look!
My eyebrows raised in surprise at this point! Obviously there's a tradeoff between convenience ("Oooh it's working! great let me deploy right now") and reproducibility ("Hmm I need to re-deploy what my teammate was working on because <reason>, let me go and find their notebook... ah here it is, oh weird why do I have this undefined variable") and this is hard on the convenience side.
Was it surprising to you too that you'd have users who wanted this location on the continuum?
> Was it surprising to you too that you'd have users who wanted this location on the continuum?
I’m still working on the “have users” piece, but there were signals for wanting more simplicity/convenience in conversations that I had with data scientists. I also felt as a data scientist that existing deployment solutions sometimes felt over-engineered for flexibility and reproducibility, at the cost of convenience. I'm exploring the other end of the continuum with Convect -- how opinionated can we get with ML workflows to simplify and still be useful?
The workflow looks somewhat like this —
Within a notebook, you can: 1) train a model, 2) deploy a model, and 3) get a shareable serverless API endpoint.
Outside the notebook, app.convect.ml provides a UI for editing, sharing, or deleting deployed models as well as getting the API key for deploying models.
The variety of data problems you can work on with good old logistic regression (or the other models in scikit-learn) is really quite vast and seemed like a reasonable place to start.
Trying to deploy a model to Azure Functions didn't play nice because of my file sizes.
Convect, in its current state, is still limited to small models (e.g. < 0.5MB). This is because the deployment happens by posting to a REST API, which hasn't yet been tested for large payloads.
I wrote up some tips for decoupling a large model from the data that's causing it to be large (https://convect.readme.io/docs/news-topic-model). However, it sounds like you're asking about a truly large model. Convect is still in the MVP stage right now, but we plan to handle larger models. 3-5 GB models will likely be feasible in the future.
Is there support for including custom code as part of the deployment?
Models trained on tabular data (which are the majority of use cases in industry) usually require some pre-processing before feeding the model (i.e., feature engineering).
I’m not a fan of deploying my models on your infra, but maybe that’s an opportunity for you. I’d pay a subscription if the tool would let me deploy to my aws account. I think you can sell a cloud formation template through the aws marketplace as a manageable way to execute that.
Anyway, looks awesome and definitely scratches a real itch. Good luck
The infra aspect has been on my mind — how to easily onboard users who prefer to use their own infra. Love the idea of selling a CF template on AWS Marketplace. I’ll look into that. Really appreciate this feedback :)