Hacker News Reader

Comment by albertzeyer | Hacker News Reader

How does it compare with TensorFlow (graph-based, i.e. symbolic) or Theano? Or maybe also JAX, which you could also see as symbolic.

In the paper it is briefly mentioned that TensorFlow/JAX are slower due to more overhead? I doubt this is true. Also, TensorFlow could be compiled to TF-lite or XLA and then C++ or whatever you like. Same for JAX.

It is also mentioned that TensorFlow/JAX perform poorly on second-order optimization. But this is not true.

Further, it is said that TensorFlow/JAX have poor performance for sparse matrices. While the performance is not great, I am not sure that other frameworks would perform faster.

Some fair benchmarks would be nice.

But then, despite just benchmarks, also a more direct comparison on a conceptual level would be nice. Because from the description, I don't directly understand how it is much different.

View on HN

Part of the motivation for SymForce was in fact serious attempts (multiple months) by us to do similar things with TensorFlow XLA and being stymied by some of these effects. Many of the problems we deal with, and many problems in robotics in general, deal with small "tensor sizes", where dimensions of individual variables are often <10, and batch sizes aren't huge either. SymForce-generated C++ code is typically one machine instruction per scalar per "op" - compiling TF / JAX etc using methods like you mention definitely helps a ton, but it doesn't reach that level of efficiency for small problems. And computing Hessians for second-order optimization tends to blow up op counts even more.

Similarly, SymForce examines sparsity of individual matrices and jacobians at code generation time, so that operations with sparse matrices are similarly one instruction per op - at runtime there's no looping or indirection through sparse data structures or allocations of sparse matrices in the optimization loop. There's definitely a cross-over point where dimensions become large enough that TF / JAX / etc will be the tool to use - we'd love to provide some more benchmarks and comparisons here. For the first two experiments in the paper (see Tables II and III), we do compare to JAX and amortizing across a batch size of 1000, SymForce is still way ahead. We tried to be careful, but the code is available so please come beat up our numbers.

> amortizing across a batch size of 1000

This put the problem outside of L2 and thus is not a fair comparison, especially if your problem is small enough to fit in L1.

In addition, XLA does a poor job in optimizing for CPU cache reuse, so I'm not surprised that directly transcoded C++ code would be much faster than XLA.

However, TensorFlow code can directly run on GPU. SymForce cannot.

To be clear, we tested a large set of batch sizes from 1 up to ~1e8. The smaller the batch size, the less competitive JAX was. The crossover point for JAX to be faster was amortizing across a batch size in the tens of thousands I believe. The ideal batch size for some of our applications is one to five. Please see symforce/benchmarks/notebooks for some of our timing.

SymForce is not trying to compute on large data-parallelism, but it could generate an inner kernel for that. In that way it complements TensorFlow nicely.

> TensorFlow code can directly run on GPU. SymForce cannot.

I think this is just a matter of tweaking the code generator, and not a hard limitation of Symforce.

Reply on news.ycombinator.com