Hacker News Reader

Comment by a2code | Hacker News Reader

This may be related to TinyML. Consider the ESP32 that introduced WiFi to MCU making it extremely popular. Is there already a comparable MCU+AI popular chip? Or will it not happen with AI but some other future technology concept?

View on HN

There are actually tons of chips that are great for this type of workload. You can run simple vision applications on any 32 bit MCU with ~256kb RAM and ROM.

There's a list of MCUs here:

https://docs.edgeimpulse.com/docs/development-platforms/offi...

And some accelerators here:

https://docs.edgeimpulse.com/docs/development-platforms/offi...

This is just stuff that has support in Edge Impulse, but there are many other chips too.

Thanks. Let me be more specific. The ESP32 included WiFi on the same chip. Is there an MCU with on-chip features for AI? Perhaps an optimized TPU combined with an MCU. Would that be an advantage?

There are a range of ML acceleration possible on existing chips. The basic 4-wide 8 bit integer SIMD extensions in NEON is available on basically all ARM Cortex M4F chips, which is already available 8+ years. It gives 4-5x speedup for neural networks.

The more recent ESP32-S3 has operations with up to 10x speedup, see https://github.com/espressif/esp-nn

Then there are RISCV chips with neural network co processors like Kendryte K210.

ARM has also defined a new set of extensions for NN acceleration, and reference designs for cores being ARM Cortex M85. Chips are becoming available this year. ST has announced they will have accelerators in several lines. There are dozens of startups creating accelerator designs and trying to pair them with MCUs.

So we have a bit already, with much more to come in the years to come.

Thanks for the reply. I did not find public documentation for Kendryte, only a Github repository. At least the code is in English. But the AI examples include an "nncase" library which I could not find on the repository. So I could not see the instructions their accelerator has.

On the other hand, esp-nn seems to be code for the xtensa instruction set. I briefly overviewed the instructions. They seem optimized for DSP rather than ML applications. Searching for SIMD returned no arithmetic instructions. Searching for parallel returned instructions for multiply and accumulate. Further, the FPU does not compute any kind of 16-bit floating point numbers.

>ARM has also defined a new set of extensions for NN acceleration Can you provide some more info about this?

The latest extensions from ARM are codenamed Helium, and they are an extension on the previously mentioned NEON. Both NEON and Helium are quite simple vector extensions, and yes it is also used for classic DSP stuff. I believe Helium also supports fp16, though for inference on MCUs I believe that int8 will continue to dominate. Here is book on Helium from ARM that seems informational https://github.com/arm-university/Arm-Helium-Technology

There is another chip that is generally available, that has a CNN accelerator/co-processor - the MAX78000 https://www.embedded.com/hardware-conversion-of-convolutiona...

Thanks again. I have to correct my previous reply. The ESP32-S3 has an extended instruction set detailed in the technical reference manual. These include vector operations (8, 16, or 32 bit).

I'm curious, why do you believe int8 will dominate?

NXP's new MCX N94x and MCX N54x microcontrollers both have dual Cortex M33 cores and an integrated NPU. Eval boards should be available for purchase any day now.

Reply on news.ycombinator.com