Hacker News Reader

> amortizing across a batch size of 1000

This put the problem outside of L2 and thus is not a fair comparison, especially if your problem is small enough to fit in L1.

In addition, XLA does a poor job in optimizing for CPU cache reuse, so I'm not surprised that directly transcoded C++ code would be much faster than XLA.

However, TensorFlow code can directly run on GPU. SymForce cannot.

View on HN

To be clear, we tested a large set of batch sizes from 1 up to ~1e8. The smaller the batch size, the less competitive JAX was. The crossover point for JAX to be faster was amortizing across a batch size in the tens of thousands I believe. The ideal batch size for some of our applications is one to five. Please see symforce/benchmarks/notebooks for some of our timing.

SymForce is not trying to compute on large data-parallelism, but it could generate an inner kernel for that. In that way it complements TensorFlow nicely.

> TensorFlow code can directly run on GPU. SymForce cannot.

I think this is just a matter of tweaking the code generator, and not a hard limitation of Symforce.

Reply on news.ycombinator.com