Show HN: Platform to serve ML/DL models at low latency

I've always had a hard time deploying ML models into production. The traditional approach is to use Flask, Gunicorn with Nginx. This requires a lot of setup time. Also, inferring model with flask is slow and requires custom code for caching and batching. Scaling in multiple machines is also hard. We have created panini.ai as a solution.

https://www.panini.ai/ is a platform to serve ML/DL models at low latency and makes it super easy to deploy AI models in the cloud. Once, deployed in the cloud it will provide you with an API key to infer the model. Our backend is written in C++, which provides very low latency during model inference and the model is stored in Kubernetes so, it is scalable to multiple nodes. We take care of caching and batching inputs during model inference.

I have also created a YouTube tutorial on how to use panini: https://www.youtube.com/watch?v=tCz-fi_NheE&t=

Please let me know what you guys think. If you have any questions, please email me at avin@panini.ai