BeagleY-AI: a 4 TOPS-capable $70 board from Beagleboard

Agreed! What I really want to know from any hardware accelerator is:

* What can it accelerate?

* What software supports it (and what's the state of driver support)?

* How much faster or more power efficient is it vs CPU/GPU?

* How exactly do I use it? (Okay, I buy the thing, it arrives in a box, I plug it in... what exact steps do I take to get to running real workloads on it?)

4 TOPS is last-gen smartphone territory - you aren't going to have a good time running big generative models, but it's a useful amount of performance for typical edge workloads like machine vision and robotics.

One thing that comes to mind is AI detection in Frigate https://docs.frigate.video. I’ve not played very much with AI but I’ve dabbled with Coral this past month and it would definitely be nice to have a fully integrated machine that could be dedicated to my security video feeds.

For scale, Microsoft requires 40 TOPS to certify a (consumer) computer as Copilot capable.

https://www.techpowerup.com/320933/microsoft-copilot-to-run-...

Probably good enough to run openCV and turn it into a people and object recognizing brain for cameras but not sure.

4 TOPS is the same as the Coral Accelerator. Look on YT for how people have used it. Another poster mentioned video object recognition with Frigate.

The bigger question for me is if it's easy to uses or add the accelerator memory to your application. For something like a virtual assistant or something with some basic image recognition, it can be really confusing about how to take advantage of the different hardware tools that are there. This is especially true in the maker space, when you're trying to leverage open source tool kits and don't necessarily have the time or knowledge to write a full, lower-level driver for open source tools kits.

Looks like 1/10 of TOPS and 1/3 of mem bandwidth of Jetson Orin Nano (the TI chip seems able to run 3733MT/s at 32 bit, so 15GB/s) but much cheaper.

The first PC where I installed Slackware 2.0, had 16 MB of RAM....

Not a powerhouse indeed.

Plus, they have way too little memory for any meaningful LLM workloads that go beyond simple inference tasks or running yolo.

Grossly underpowered compared to a fat stonking GPU, yes. But the whole BeagleY-AI board probably consumes less power at full tilt than just the fans to keep a modern 2-slot NVIDIA GPU cool.

For embedded applications, the C7x DSPs are mighty. Lots of existing C6x DSP code can easily be recompiled to target the C7x with minimal effort as well, which is a big deal. Making your C6x code work efficiently on the C7x may require more than just a recompile, but being able to leverage existing C6x DSP codebases with minimal investment while getting the extra performance of the C7x is a very big deal.

I don't think a quad-core ARM Cortex-A53 with a pair of C7x DSPs is trying to compete with an NVIDIA Hopper. But if you're making a $100 embedded product that has to handle video, this seems quite attractive.

What do the cool kids use these days? I know YOLO is still relevant but I’m not sure what’s the new stuff.

https://openbeagle.org/beagley-ai/beagley-ai - at a glance, it looks like there's everything you need to be able to fab your own boards, if you really wanted to.

Well it's certainly a good idea to get a better understanding of potential purchases. Unmotivated impulse purchases just clutter up your mind, your house and are bad for the environment.

"If you have to ask" you're almost certainly better off with a rasp5 or probably 4 if you are looking for a mini PC to get some use out of that monitor you have flying around. Raspberry has a larger community and is therefore more accessible when starting out.

If you're looking to get a better understanding of the "low-level" workings of computers, consider an Arduino.

Similar but Beaglebones have flash drive controller which they boot off so they do better at not corrupting themselves on power cycling. I don't know about the BeagleY-AI but the Beaglebone black's power sequencer has won't fix issues that cause the board to fail to start if the power comes on too slow.

Open source hardware means they give you schematics and the board layout as an Altium project. You can take that and customize it to fit your needs. When it comes to processors that can run embedded linux that's actually common. Advantage if you don't modify the layout of the high speed buses your prototype will likely just work.

> Quad-core 64-bit Arm®Cortex®-A53 CPU subsystem at 1.4GHz

It's slower.

> Memory 4GB LPDDR4

Has less and slower memory.

> Arm Cortex-R5 subsystem for low-latency I/O and control

Might use less power at idle, the Pi 5 is extremely power hungry when doing nothing (~5W).

> Dual general-purpose C7x DSP with Matrix Multiply Accelerator (MMA) capable of 4 TOPs

An AI chip with questionable software support.

The rest seems about the same, it's loosely based on the same PCB layout.

Over 10 years later, I can say that the original BeagleBone was far more reliable than the original Raspberry Pi. The Texa Instruments specifications & documents for the processor were far better as well. I've had several Raspberry Pis just randomly die, but every one of the Beagle Bones are still going. I've heard similar feedback from people with IOT startups. RPis work great until some just randomly die.

Today I find the quality of the Raspberry Pi is dramatically improved, the tooling to help a beginner get started is amazing, and of course there's just simply a much larger user base that have likely resolved the problems you'll encounter.

However the real magic of a Beagle bone is listed in this features: Arm Cortex-R5 subsystem for low-latency I/O and control

There are many use cases such as robotics, where there is a need real time control and Beagle bones have this capability built in, whereas with Raspberry Pi you'll see people connecting an Arduino and a Raspberry Pi together to meet this requirement. Its kludgy, burns tons of time, and it also adds up cost wise.

In any case if you're just starting out, and you ever get interested in these things, know that you'll likely end up with many of them over the years. RPi is great for beginners, but if I was deploying something into the field as an IOT startup etc, I still prefer a Beagle.

My 2c

The main difference is the Cortex-R5 subsystem. Beaglebone, the famed predecessor, had PRUs. These features paired with a powerful Cortex-A processor are really useful if you have a complex program that needs to do something (IO/control, for example) with a very high precision time-wise. If you have high punctuality demands but a rather simple algorithm, you can simply use an Arduino or Raspberry Pi Pico (RP2040). If you just have heavy load, you can use Raspberry Pi. One common use for Beaglebone was (is?) 3D printer control.

As far as I am concerned, the AI acceleration in BeagleY-AI's DSP is just a fad (TI DSP itself is quite respected in the industry).

I would probably say it's better to buy a Pi 4/5/Zero and one of RP2040/Arduino/micro:bit/Nucleo and master them separately, possibly with a UART/I2C/SPI link in between before moving on to a single package.

Doesn't PCIe mean you could trivially add support for NVME though via an extension board?

MLCommons provides MLPerf which is a series of benchmarks anyone can run and compare results. For devices like the BeagleY-AI the "edge" category provides a series of various benchmarks to run covering things like object detection, speech recognition, and now smaller LLM's.

https://mlcommons.org/benchmarks/inference-edge/

reminds me of the "multimedia PC" of the 90s. But at least that branding didn't enforce cloud services and Microsoft telemetry

Ok Microsoft, let's add a copilot key to our keyboards.

But can we then remove the Windows key? unclear...

And it's not honest work either. LPDDR4 is now more expensive per GB and in smaller packages than LPDDR4X, plus it uses twice as much power. It's a lose-lose-lose situation not to use it.

You are confusing programmable shaders and GPGPU with AI.

AI only needs matrix multiplication and activation functions to be accelerated. For everything else the GPU is already so fast there is no point in further improvements.

Given the recent HN post about an update to the OpenVMS project I thought the same thing when I saw "TOPS-capable." Hmm, maybe some kind of port from long-retired propeller heads at DEC/Compaq/HP.

Nope. Looked up "4 TOPS" and learned the famed Motown group was still performing though some of the original members had passed.

Finally determined that "TOPS" did indeed mean "trillion operations per second."

Wasn't so long ago these types of things were only found inside some mountain laboratory surrounded by heavily-guarded 5 mile restricted access perimeters.

~$70.00 sometime in June 2024. Don't tell me the industry isn't still sprinting all-out 54 years after TOPS-10 was first introduced.