Comment by michaelt | Hacker News Reader

Comment by michaelt | Hacker News Reader

Parent Full thread

According to [1] the manufacturers claim 2% of the TOPS of a RTX 4090, and only 0.8% the power consumption.

$200 in prototype quantities [2] which is 12% the price of a 4090 - but perhaps the price drops when you order them in bulk?

They claim it compares favourably to an 'Nvidia Xavier NX' for image classification tasks, providing somewhat more FPS at significantly lower power consumption. 218 fps running YOLOv5m on 640×640 inputs.

They're completely silent about the amount of memory it has, but you can fit int8 YOLOv5m into about 20 MB so it'll certainly be an amount measured in megabytes rather than gigabytes.

Their target market is "CCTV camera that tracks cars and people" rather than "Run an LLM" or "train a network from scratch"

[1] https://hailo.ai/products/ai-accelerators/hailo-8-ai-acceler...

[2] https://up-shop.org/hailo-m2-key.html

From what I know it doesn’t have memory but streams everything to the chip. So there’s no limit regarding the size of the the neural network (unlike Google Coral)

Maybe? If a 20 MB network achieves 218 fps that'd need 4.3 GB/s of bandwidth just to stream the network, completely ignoring the images. And they use PCIe Gen3 which is 1 GB/s per lane, with different products having 1, 2 or 4 lanes.

So... maybe just about?

Some of their marketing mentions no need for external DRAM, for example, from this CNX article[1]:

"One of the key reasons for the performance improvement is that RAM is self-contained without the need for external DRAM like other solutions. This decreases latency a lot and reduces power consumption."

Not sure how much RAM is included on the chip, but I'm also thinking in the tens of MB range, certainly not gigabytes.

[1] https://www.cnx-software.com/2020/10/07/learn-more-about-hai...

PCIE is slower than DRAM, so not having DRAM is not a performance improvement, but a hardware limitation reducing the scope of usability. Though it doesn't mean bad - cheaper to produce, serving its own use cases.

Process a stream of data - yes. Machine learning on large set of data - no.

I think what's implied by their marketing at least is that DRAM is included on the chip itself (unlike other solutions like GPUs with external RAM chips)... but the problem is that limits the amount of available RAM quite severely.

Reply on news.ycombinator.com