TensorFlow 2.0 is coming | Hacker News Reader

TensorFlow 2.0 is coming

groups.google.com

129 pointsalbertzeyer32 comments

> Since the open-source release in 2015, TensorFlow has become the world’s most widely adopted machine learning framework

I'd wager that non-trivial (non-tutorial?) usage of scikit-learn is significantly higher.

'computation graph framework' isn't as sexy, I guess

The #1 thing I would love to see in future versions of Tensorflow is a better C API, with documented support for gradients, which are essentially for porting Tensorflow to other languages.

As someone who is not very familiar with Python, I found it very difficult to port code over, and so I've mostly paused my porting effort for the time.

Other than that, I quite like Tensorflow, and intend to use it more and more as time goes along.

Amen, especially for the training side. Python and C++ leave a lot to be desired. Extrapolating from this quote:

> Support for more platforms and languages, and improved compatibility and parity between these components via standardization on exchange formats and alignment of APIs

I hope that means a solid C API, but it might also mean higher level e.g. protobuf/grpc or something.

Definitely. C is the Rosetta Stone of ABIs. Python is simply not suitable for that purpose. It is compatible with, well, the same version of Python.

C++ has the same problem. It's even worse in a sense, since there's still effectively no compiler-independent binary object model at all.

I don't think this is possible. At least in TensorFlow 1.x, a lot of non-trivial parts (other than Operators and Gradients) are implemented in Python directly.

See their Java bindings, the functionalities exposed are miles far from the Python version.

I feel like I can see the fingerprints of pytorch on the direction TF leadership is choosing to go. "Eager execution" and "easier to learn and apply" are where pytorch steals market share from TF, resulting in moving towards a more "dynamic" graph model.

Eager Execution was added to TensorFlow a couple versions ago.

https://www.tensorflow.org/guide/eager

And the parent's point still stands.

As someone joked to me, Google has too many compiler people, and this may have explained some design choices.

While I am sure pytorch competition contributed to this change, having a graph-based API was a terrible idea whose time would have passed anyway I think. It made some many things complicated for users. As an implementation/optimization strategy, using a graph was a reasonable choice, but to let drive the API as it did was indeed terrible.

It is reassuring to see the TF team addressing the main flaws of the library: removal of graph-based API, and fixing the lack of consistency/plethora of APIs attacking the same problems.

Isn't "easier to learn and apply" usually code for "slower and more difficult to optimize"?

Not in the case of PyTorch. Massively easier to deal with _and_ faster for just about everything.

The pytorch graph model is not as dynamic as you think it is, because batches. All batch elements must be processed by the exact same code trace, limiting per element dynamism. The dynamic aspect of pytorch is an implementation detail for delivering autograd over a trace of tensor operations.

What pytorch does very well is to play nice python/numpy semantics, especially via broadcasting[0]. There is very little cognitive overhead between a pytorch program and its equivalent python/numpy representation.

What tensorflow does very well is to execute computational graphs on a wide variety of backends. One gpu, many gpus, distributed gpus, servers, mobile, browsers. The recently announced autograph[1] merges the clarity of python/numpy coding style with tensorflow execution engine, offering autograd via compile-time abstract interpretation over all possible traces.

[0] https://docs.scipy.org/doc/numpy/user/basics.broadcasting.ht...

[1] https://www.tensorflow.org/guide/autograph

Multi GPU in PyTorch is usually just a few lines of code. Can’t be any easier. Wrap model in DataParallel and enjoy life. Last time I touched TF, things were pretty gnarly for this frequent use case. Don’t know how it is now.

The removal of contrib is surprising but probably necessary. It will be interesting to see which projects get integrated vs removed. I'll be watching bayesflow and keras.

Isn't bayesflow already a separate project (i.e. tensorflow-probability[1])?

[1] https://github.com/tensorflow/probability

Keras was integrated quite a while ago IIRC.

AMD Graphics card support please?

There was a OpenCL branch no long ago that was supposed to make it into mainline at some point. You can (kinda) track progress of that here: https://github.com/tensorflow/tensorflow/issues/22

We are (well, AMD is) working on it. It's all being done in the open; there are PRs you can follow. You can even help out if you're so inclined.

It's not going to be simple or fast, but it's coming...

(I work on XLA, a compiler for TensorFlow, and I've been working closely with AMD on the TF/XLA -> AMDGPU port. My team also works on CUDA support in clang, and we've been reviewing AMD's patches to support HIP in upstream clang.)

https://github.com/ROCmSoftwarePlatform/tensorflow-upstream

I wonder what new language bindings they will support. I have experimented with saving Keras models, converting to Racket Scheme, and writing a runtime. It would be way better to have it officially supported.

Haskell support has worked OK for a while. The languages I would most like to see supported are common implementations of Common Lisp like SBCL and Clozure. I think there is some future for hybrid connectionist and symbolic AI and it would thrill me to have Common Lisp support for TensorFlow.

On a related note, there's a Clojure API for MXNet (another major deep learning framework).

https://mxnet.incubator.apache.org/api/clojure/index.html

Thanks, that looks very good. Unfortunately Clojure and I don’t really click. I have about two years professional experience with Clojure but except for my site http://Cookingspace.com I don’t much use Clojure for personal projects. I prefer Common Lisp. For functional programming, I would substitute Haskell for Clojure except my Haskell skills are so-so.

Interesting, what do you mean by saving Keras models and converting to Racket?

Saved Keras model meta data and weights, wrote Racket code to read weights, use low level matrix library, relu , sigmoid, to use trained models. So, just use TensorFlow for training.

This works, but the performance was not great. I want something more robust and performant.

Any chance of being able to build models in Go (instead of just running them) in Tensorflow 2.0?

I just hope that it's easier to build from source. We need to support multiple TF versions, and since we have shared storage, we don't install relative to system root, we install relative to a TF-version-specific root path (like /software/open/tensorflow/2.0). That's been fairly annoying to do on CentOS.