Coral USB Accelerator with Google's Edge TPU
coral.ai
coral.ai
But even CPU inference is both faster and more energy efficient with a modern Arm SBC chip, and things like the Hailo chip are way faster for similar price, if you have an M.2 slot.
I haven't seen a good USB port alternative for edge devices though.
The big problem is Google seems to have let the whole thing stagnate since like 2019. They could have some near little 5/10/20 TOPS NPUs for cheap if they had continued developing this hardware ecosystem :(
Google's flightiness strikes again. How they expect developers (and to some degree consumers) to invest in their churning product lines is beyond me. What's the point in buying a Google product when there's a good chance Google will drop software support and any further development in 5 years or less?
It's only a few weeks ago that they added support for them in the Home app.
I believe they have a 802.15.4 radio, maybe the x chipset is too old, but it would have been great to get Matter over Thread support for them.
New-comers like Aqara will instead take up that space.
> As per https://www.tensorflow.org/guide/versions , Can we assume that libedgetpu released along with a tflite version is compatible with all the versions of tflite in the same major version?
> Hi, we can't give any guarantee that libedgetpu released along with a tflite version is compatible with all the versions of tflite in the same major version.
But even CPU inference is both faster and more energy efficient with a modern Arm SBC chip
* https://www.reddit.com/r/frigate_nvr/s/ncxP1YQDfB
* https://github.com/blakeblackshear/frigate/blob/e773d63c16d9...
* https://liliputing.com/rubik-pi-is-a-compact-dev-board-with-...
* https://www.cnx-software.com/2025/01/09/qualcomm-qcs6490-rub...
My needs are pretty much the same as people who buy camera bundles from big box stores. I want reliable motion detection for intruders, deliveries, and visitors, and the ability to watch videos recorded in the past couple of weeks.
I recently got a few to try out, and expressly chose them because they do motion and sound detection on device and also support microSD for local recording.
I've only had them a few weeks so can't speak to any slow-showing pain points, but so far both the video doorbell[1] and two inexpensive cameras I purchased[2][3] to test out have been awesome.
I set up an automation so they record continuously when I leave and when home to record on detection only (motion for all three and sound for the Kasa camera) to try to be economical on wearing out the SD cards. But for me personally knowing I'll likely have those go out on me and need replaced was an ok trade off for the convenience, and probably a wash financially because everything I wanted happens locally whereas I kept seeing them gated behind a subscription plan when looking at other options.
There's also an option in the app to enable them to speak stream locally to a NAS or NVR via RTSP if you want to do that with them. So I can eventually set that up for more reliability when the eventual SD burnout occurs, and scratch my tinkering itch with things like streaming it to Frigate and testing it out vs the native detection features, without any actual presasure to need to since it all just works as is.
The doorbell is what I was originally needing the continuous local recording and on-device object detection for. The two cameras were bonuses I threw in to grab a few of their inexpensive models to try out while I was at it. And so far for about $100 in total I've been impressed. Key word being so far – they're still recent enough I might be in my honeymoon phase with them and just don't know it yet.
[1] https://www.amazon.com/dp/B0CQQZZXH9
I wish the camera could stream to Frigate/whatever but stream empty delta frames when nothing is detected.
I got repurposed HP G2 SFF desktop with old i5-6500 cpu running proxmox with bunch of VMs and LXC containers including frigate.
I am passing both coral USB through to frigate container for object detection and passing intel's gpu through for video decoding.
With 10 cameras continuously recording, corals inference cpu usage is about 12%, frigate CPU usage is about 5%, although a service called go2rtc which is used by frigate to read the cameras streams and restream them to frigate takes up about 15% of the cpu.
Overall my cpu usage fluctuates below 30% on that entire machine devoted to more than proxmox.
I did run the watt calculation on that machine and it was something reasonable, dont recall it right now
If you want home surveillance, you can just tie a bunch to ethernet and they'll do on-device AI.
* https://www.pishop.us/product/raspberry-pi-ai-hat-13-tops/
Even the bigger Hailo-8 hat with >6x the compute of the Coral is only $135.
* https://www.pishop.us/product/raspberry-pi-ai-hat-26-tops/
Thunderbolt 4
There are a couple solutions, but then you have to have hardware that has a Thunderbolt port as well... and those aren't everywhere, especially on cheaper computers and SBCs.
This is not something that is very useful or relevant these days - it's basically abandoned at this point and only works with older versions of Python etc.
Searching around, it appears that the Coral USB Accelerator does about 4 TOPS.
The Raspberry Pi 5 AI Kit which costs a few bucks more (and came out last year instead of in 2020) does 13 TOPS.
The Jetson Orin Nano Super, which costs $250, does 67 TOPS, and was just updated last month (although it's a refresh of the original product).
I own all three of these products and they are all very frustrating to work with, so you need to have a very specific use-case to make them worthwhile - if you don't, just stick with your machine's GPU.
FWIW, there wasn't actually a physical revision/refresh, it's all software. So older owners can just update and get the boost as well.[0]
> With the same hardware architecture, this performance boost is enabled by a new power mode which increases the GPU, memory, and CPU clocks. All previous Jetson Orin Nano Developer Kits can use the new power mode by upgrading to the latest version of JetPack.
[0]: https://developer.nvidia.com/blog/nvidia-jetson-orin-nano-de...
See my other comment regarding efficiency of my Intel Xe iGPU.
Jetson is a different league though. These can run even LLMs (tho 16 GB version was overpriced when I bought during covid, so went for 8 GB). Ollama Just Works (tm); now compared to getting Ollama working with ROCm on my 6700 XT however, that was frustrating.
So, object detection with Tensorflow, works well w/these Coral TPUs. However, you can forget even running Whisper.cpp
One nice thing the Coral USB has for it though is it is USB. You can get it to work on practically any machine. Great for demos.
For old version of Python fire up a VM, OCI, use a decent package manager like uv or pipx.
https://github.com/jveitchmichaelis/edgetpu-yolo
You really do need the model to be defined in Tensorflow though. Channel ordering screws things up in PyTorch and it's not trivial to get round it, otherwise the compiler does all sorts of weird gymnastics to permute for you.
Sincerely, Miles' council
RK3566 boards are cheaper than a Raspberry Pi 5, but you get less model support, and no Frigate+ model.
The mini-PCIe variant is much more reliable, but I ended up ditching the Coral entirely and replacing it with a GTX 1060.
My main reason for getting one was power efficency compared to a traditional GPU for this task.
Sounds cool but for me that would mean more upgrades for that server.
Coral runs on 0.5 watts , which is way less than the geforce 1080 i used before.
Idle is IIRC 0.01W.
In terms of longevity - it's kinda table stakes in the electronics industry. Nobody's going to integrate your edge AI chip into their smart CCTV camera or whatever if you can't promise 5+ years of availability. There are chipmakers still selling parts they launched in the 1980s.