Hacker News Reader

Comment by minimaxir | Hacker News Reader

> What's different now ?

PyTorch adoption: back when Keras went hard into TensorFlow in 2018, both TF and PyTorch adoption were about the same with TF having a bit more popularity. Now, most of the papers and models released are PyTorch-first.

View on HN

Yes I understand why they do the move (they want to attract pytorch user). What's the benefit for the user instead of directly using pytorch for example ? I see we can maybe use tpu by switching to jax etc...

PS: sorry I'm a bit salty by my user experience of Keras.

Pytorch is an animal by itself when you try to put it into production. They have started addressing it with torch 2.0 but it still has lengths to go. With this you can switch to TFserve if you have usual architecture.

You can just use Triton which is basically TFserve for Tensorflow, Pytorch, Onnx and more.

Can you explain that?

My understand of Triton is more that this is an alternative to CUDA, but instead you write it directly in Python, and on a slightly higher-level, and it does a lot of optimizations automatically. So basically: Python -> Triton-IR -> LLVM-IR -> PTX.

https://openai.com/research/triton

It's confusing, there's OpenAI Triton (what you're thinking of) and Nvidia Triton server (a different thing).

Original comment is referring to Nvidia triton inference server

Tensorflow has some advantages, like being able to use tf-lite for embedded devices. JAX is amazing on the TPU, which AFAIK pytorch doesn't have great support for.

I assume most people will still research in PyTorch, but then move it over the Keras for production models if they need multi-platform support.

Keras has a cleaner API compared to base PyTorch, especially if you want to use the Sequential construction as demoed in the post.

How so? You can use torch.nn.Sequential pretty much equivalently?

https://pytorch.org/docs/stable/generated/torch.nn.Sequentia...

Huh, didn't realize base PyTorch had an equivalent Sequential API.

The point about the better API overall still stands (notably including the actual training part, as base PyTorch requires you to implement your own loop)

Some people would say that this is an advantage of PyTorch, that it is very easy to write and play around with your own loop.

But if you don't want that, if you want to go a bit higher level w.r.t. training, there is e.g. PyTorch Lightning on top of PyTorch.

If you say the API is better in Keras, can you give any examples? They look kind of similar to me. Keras looks a bit more complicated, esp when looking at the internals (which I tend to do often, to better understand what's actually happening), which is a negative point.

Adding keras abstractions on top of pytorch seems like negative value-added to me. It only barely abstracts more the easy/mundane stuff, while creating yet another layer of indirection between ideas and machine code in the current Rube-Goldberg mess that people call modern ML.

Keras has always been "lipstick on a pig for tensorflow". Its value beyond that seems tenuous at best.

Reply on news.ycombinator.com