Hacker News Reader

Comment by lagrange77 | Hacker News Reader

Wow! After repeatedly unsuccessfully tying to get an overview over NN accelerators today, i just found this on the HN homepage. Looks very promising and to me this seems to be a very logical approach, in terms of efficiency (besides analog computers).

I would also be very interested in some benchmarks comparing the generated hardware with things like Google Coral or Nvidia Jetson.

I am sure this will be a success.

View on HN

Glad this helped clarify things for you! The tricky thing about benchmarks is that one of the key benefits of Tensil is the flexibility to find a trade-off between performance, accuracy, cost and power usage that works for you. Benchmarks that only consider performance or performance per watt can be a bit narrow from that point of view. That said, this is a good idea and we'll add some comparisons that we think make sense to the docs!

I wanted to add something about the xilinx dpu, and you brushed on the subject but I was quite unhappy with the softip thing. It embeds all instructions for all kinds of networks so, taking a lot of gates for unused features, it's not much customizable, and perf for anything else than vanilla conv2d stuff quickly gets down. Buying an Alveo board to get such low inference perf was a gutpunch.

FINN seems far better there. At least you get millions inference/sec on simple quantized CNN1Ds.

The xrt api is simple and relatively ok, too. Stream data, execute inference, fetch results, mostly sync, so you have to wrap a lot of threading there, but the basics are there.

Yep, this is something we've heard before. If you're really familiar with the Xilinx ecosystem, one way we've described Tensil is that it is the "Microblaze for ML" - easy to use, lots of flexibility and customizability, with performance good enough for most applications. The DPU and FINN would then be the more specialized tool for situations where you need specific features they are optimized for.

Ha, now you've made me curious. Let's see how everything progresses then. Thanks for the earnestness on these comment threads.

You're very welcome! Stay in touch - I've listed some contact methods here and there in the thread, and we'd love to hear from you again.

Thank you!

One more thing to keep a look on in the NN accelerator world is TensTorrent. That thing looks amazing, but it's mostly for datacenter and 'heavy' edge (pcie board, at least 75W, so to measure against Alveo U50/U55 and Tesla T4/A30 and up to 300W so A40/A100).

Thanks, we'll take a look!

Reply on news.ycombinator.com