Hacker News Reader

Really cool writeup! I'm surprised you trained with TF and then deployed with PT. I feel like PT is considered easier to train with while TF is easier and faster to set up production-level serving. I think you might have had to do less work if you used TF Serving.

View on HN

https://github.com/huggingface/transformers (was) based on PyTorch, so they originally had lot of the models like gpt2-small. Because of that influence, PyTorch was probably going to win there.

The one con about TF Serve (TFX) is packaging the entire model into the container (so that ends up being 3gb?+). This was a couple of months ago, so I might be wrong by now ... It was an area I wasn't very confident I could do, TF Serve is really new, so there aren't many guides (and many were already out of date).

TF Serving has been around several years and is pretty mature. (It predates TFX). You can stick the model in the container or have the container pull the model from blob storage (or have no container at all if you really want).

It's too bad you didn't find a good guide - if you have the training dump a SavedModelBundle at the end, you can have a production-quality serving microservice up and running in about two lines of code - https://www.tensorflow.org/tfx/serving/docker.

But it doesn't really matter since you got it working.

Reply on news.ycombinator.com