Show HN: Deploy Scikit and Keras Models with a Simple Drag and Drop | Hacker News Reader

Show HN: Deploy Scikit and Keras Models with a Simple Drag and Drop

inferrd.com

25 pointstheo3111 comments

Is this able to support more than 50 requests per second? Are there any benchmarks on performance overhead of the underlying web server/routing that is handling the requests?

Is there any reason that you select 50 requests per second? Anyway, I'm also interested in performance metrics.

Theoretically we can support up to thousands of request per second under the entreprise plan.

We stopped at 50 request/s in the pricing table because that seemed like a reasonable number for most use cases.

It seems very interesting! What about support for pytorch models or .onnx? I usually use the pytorch->onnx->tensorrt to deploy models.

Hi! We do not currently have PyTorch model but it's one of the next items in our roadmap.

If you have a more custom pipeline, we have a custom environment where you can deploy any custom code with specific package versions!

Looks interesting! How about models that require dictionaries - e.g. tf-idf to convert text into a feature vector? Does it allow for some preprocessing?

Hi! If don’t need any pre processing we have built in support for all the major librairies.

In addition we support custom pre and post processor via custom environments! Simply write your inference code into a predict.py and we take care of the rest.

(Btw I am really into CML.dev, great idea)

I see that you accept models up to 1 GB. It seems the inference time might be high for models of this size on CPUs. Do you use GPUs to speed up inference for deep learning models ?

We don’t offer GPU accelerated inference for now. However it’s on our roadmap, sign up for inferrd to get updates!

You might want to change 100 Mb --> 100 MB

Thank you for spotting that, it's 100MB.

Reply on news.ycombinator.com