Hacker News Reader

Comment by daghamm | Hacker News Reader

I am not familiar with this NN accelerator.

Does anyone have a comparison between Hailo and, say, a mid or high-end GPU or a TPU?

According to [1] the manufacturers claim 2% of the TOPS of a RTX 4090, and only 0.8% the power consumption.

$200 in prototype quantities [2] which is 12% the price of a 4090 - but perhaps the price drops when you order them in bulk?

They claim it compares favourably to an 'Nvidia Xavier NX' for image classification tasks, providing somewhat more FPS at significantly lower power consumption. 218 fps running YOLOv5m on 640×640 inputs.

They're completely silent about the amount of memory it has, but you can fit int8 YOLOv5m into about 20 MB so it'll certainly be an amount measured in megabytes rather than gigabytes.

Their target market is "CCTV camera that tracks cars and people" rather than "Run an LLM" or "train a network from scratch"

[1] https://hailo.ai/products/ai-accelerators/hailo-8-ai-acceler...

[2] https://up-shop.org/hailo-m2-key.html

From what I know it doesn’t have memory but streams everything to the chip. So there’s no limit regarding the size of the the neural network (unlike Google Coral)

Maybe? If a 20 MB network achieves 218 fps that'd need 4.3 GB/s of bandwidth just to stream the network, completely ignoring the images. And they use PCIe Gen3 which is 1 GB/s per lane, with different products having 1, 2 or 4 lanes.

So... maybe just about?

Some of their marketing mentions no need for external DRAM, for example, from this CNX article[1]:

"One of the key reasons for the performance improvement is that RAM is self-contained without the need for external DRAM like other solutions. This decreases latency a lot and reduces power consumption."

Not sure how much RAM is included on the chip, but I'm also thinking in the tens of MB range, certainly not gigabytes.

[1] https://www.cnx-software.com/2020/10/07/learn-more-about-hai...

PCIE is slower than DRAM, so not having DRAM is not a performance improvement, but a hardware limitation reducing the scope of usability. Though it doesn't mean bad - cheaper to produce, serving its own use cases.

Process a stream of data - yes. Machine learning on large set of data - no.

I think what's implied by their marketing at least is that DRAM is included on the chip itself (unlike other solutions like GPUs with external RAM chips)... but the problem is that limits the amount of available RAM quite severely.

Hailo is one of the newer GPU startups, so not surprising that many people have not heard of them yet. So far price/performance/power consumption of the Hailo products seems to be filling a rater large gap between the Amba stuff that is very well suited for 1-4 camera streams in a typical SoC-based device, and the Jetson, which is really kind of overpriced and power hungry for a lot of video applications (at least IMO).

Very cheap and power efficient if you're willing to run in int8/int4 and have unlimited time and patience for development

It’s a competitor to Google Coral (seems abandoned) and NVIDIA Jetson. I’ve been using it for more than a year and the hardware seems to be one of the best on the market. The software (how to actually do inference on the chip) is subpar though.

Reply on news.ycombinator.com