Hacker News Reader

We use it for inference and it works fine. Now with EFS support it's even better.

We have used it successfully with both Tensorflow and Gorgonia

It depends. Do you need a GPU? If so, your options are more limited. If not, then it's definitely doable!

No GPU so ye its very bad :P

Depends on whether you need GPU for model serving. Most companies I talk to use CPU for serving and GPU for training, so it's not a bad idea

OP here, I was thinking more CPU for inference. Training happens elsewhere on GPUs.

Just mean that you'll need to deal with latency, either through provisioned capacity lambda or build it into your service SLA :)