What’s New in TensorFlow 2.10?

First, is TF's deployment has always been ahead of Pytorch's (although the gap is closing, especially as Onnx becomes more popular).

However, the more important reason is how amazingly good value TPUs are in terms of their RAM and FLOPS. Although pytorch has XLA support, it just doesn't work as well as TF on TPU pods.

In Kaggle competitions, when the input data can fit on a consumer GPU everyone uses pytorch. When it doesn't, everyone uses the free TPUs on the Kaggle platform and reverts to tensorflow/keras.

Also, I often find myself using the TF data pre-processing pipeline even from inside PyTorch, because it's just so much easier to get excellent processing performance with TF. Not sure why, but PyTorch is in many cases "unnecessarily" single-threaded.

If I'm just prototyping something myself, I usually reach for TF first, mostly because Keras feels like the "right" level of abstraction for most stuff. I used to prototype things with fastai/pytorch more, but newer developers didn't like how much was hidden behind multiple levels of *kwargs and some of the dataloader stuff could get tricky if you tried to do anything non-standard. I haven't tried PyTorch Lightning.

Besides the deployment story, which is pretty big and others have touched on, there are some minor things that feel nice in the TF/Keras ecosystem:

- Part of deployment, I guess, but I like that more of the preprocessing can happen in the model, vs as a separate step. The less transforming of data that has to be done in the serving code, the fewer possibilities for things to get out of sync and introduce bad data at runtime.

- Keras being able to infer input sizes in layers is nice for avoiding a bunch of bookkeeping code to calculate layer sizes.

That said, I feel like maybe the TF ecosystem has more sharp edges? I've encountered more than a few of them lately as I've been doing work on some recommender models using tfrs. I've also run into things like tensorboard logging not working with entire classes of layers and causing training to crash.

I'm curious - what are some things about PyTorch that make it better for almost every single use case?