Hacker News Reader

Comment by ipsum2 | Hacker News Reader

Cheaper than the $150 Google TPU Dev Board, and looks like it can do training as well as inference. Also, doesn't require you to send your model to their company. Nice!

View on HN

Cheaper than the $150 Google TPU Dev Board, and looks like it can do training as well as inference.

Training is a slight win (although it's going to be too slow for anything useful really).

But it looks like the TPU will outperform this somewhat for inference. The 256 core Jetson (this has 128 cores) could run MobileNet-v2 at between 12 and 20 ms per image (depending on batch size)[1], while the USB TPU adapter takes 2.3ms per image [2]

Also, doesn't require you to send your model to their company.

Nor does the TPU dev board.

[1] https://arxiv.org/pdf/1810.00736.pdf (see table 1)

[2] https://coral.withgoogle.com/tutorials/edgetpu-faq/

Seems to lose in Mobilenet and win in other benches according to Nvidia's benchmarks.

I'll wait for independent testing before I drop $100.

https://devblogs.nvidia.com/jetson-nano-ai-computing/

Horseshit in that bench right off the bat: I have a Google Edge TPU board right in front of me and its perf on SSD300 is 70fps, not 48. That's with the browser demo, which (as far as I can tell) includes realtime encoding of h264 for streaming. Almost twice as much as Jetson, and likely in a much more modest power envelope. NVIDIA is known for dishonesty in their benchmarks. Although TPU is, of course, a quantized play, and Maxwell will really suck for that, unless it's been tweaked specifically for this board.

OTOH, fp32 models are _much_ easier to work with, and this thing has more RAM so you can waste it on 32 bit weights, and NVIDIA's software toolkit is second to none. So the Jetson looks pretty tempting as well. I just wish they didn't try to insult my intelligence.

Oh yeah - I never said to trust their benches. Take OEM benches especially vs competitors with the grainiest grains of salt.

When people start getting their hands on them I'll start seeing independent benches, and I think anandtech got their hands on one. Hopefully soon™.

Interesting. I hadn't seen that, but the NVidia numbers on their own products seem credible. I do agree that the flexibility of having real CUDA cores is nice.

couldn't this play games then? seems to be a pretty nice platform for a homebrew videogame console.

definitely could. retroarch would run great under linux, and a wide variety of cores are available.

but im more interested as a cudann box

>> Nor does the TPU dev board.

Yes it does require you to send your model to Google: https://coral.withgoogle.com/web-compiler/

Oh, interesting! I hadn't seen that.

I suspect it's not going to be like that forever, although I could be wrong. No self respecting AI shop will send them anything production-grade if that's how they want to play it. It'll all be hobbyists and bullshit projects, which, I suspect, is not the clientele they want to attact if it's even a somewhat serious cloud/IoT play. Although this is Google, so it could be canceled in 6 months, too, after the project managers and TLs get their promotions.

Google should license this thing to someone else who can make it in good quantity and sell it really cheap, so others build it into their designs. I'd be pretty excited if that happened.

And you have to check a box that says "I agree that my model will only be used for applications that follow Google's AI Principles." Wow.

Which may or may not include automatically killing people via drones.

They're pretty clear that that isn't kosher:

Weapons or other technologies whose principal purpose or implementation is to cause or directly facilitate injury to people.

https://www.blog.google/technology/ai/ai-principles/

But drones that spy on people are ok as long as they aren't "violating internationally accepted norms" which sort of sounds like a blank check.

Wouldn't this depend on the definition of "injury"? You could argue that under some definitions of "injury" Google violates its own "AI principles" already.

Yeah, but Google Edge TPU can do faster than realtime (about 70fps) 90-class object detection with SSD300. Just plug in a webcam. I wonder how this compares both in terms of the raw throughput and FPS per watt. If it compares favorably in even one of these without being too terrible in the other, this could be pretty cool.

there is no reason to train on that thing (except for zero shot classification demos). any cheap/lower end Nvidia gpu would do a much better job, and you would then transfer the model to the embedded thing.

Would it do better than a CPU for training? I do my dev on a MacBook Air and use AWS for training, if this cheap gpu will be a few times faster than my Air CPU than I’d be willing to get it. I usually work with medium sized models/data, too big for CPU but don’t need a multi gpu cluster.

I would stick to AWS for training. Your CPU will be orders of magnitude slower than most GPUs for training.

Google colab gpu instances are free and likely faster than the jetson (and definitely faster then typical laptop or desktop cpu only training), just save the models to your gdrive

https://colab.research.google.com/notebooks/welcome.ipynb#re...

Can you recommend a cheap/lower end Nvidia GPU for training for someone who just wants to play around with NNs a bit and isn't interested in peak performance? Could I get something for $100 that would be better than my CPU?

While you can in theory run CUDA on an Nvidia 1030 GPU (currently around $100) it's really not worth it. The cheapest card actually worth buying for ML is the 1050 Ti, which can be had for around $200.

If your budget is $100 I'd take that money and hunt around for various cloud based solutions. Most have introductory offers and/or cheap/free solutions for hobbyists with modest needs. $100 will go a long way on these services if you're careful. Once you've used up your $100 you'll have a much better idea of what, if anything, you actually need.

the other options is Google's free gpu instances on it's colab notebooks https://colab.research.google.com/notebooks/welcome.ipynb#re...

that's what I would recommend for your budget probably

That's correct, you can transfer any trained model to the Nano and run it there using the NVIDIA TensorRT library/toolchain.

Quad core arm, 4gb of ram and gigE!

Seems like a really nice board

Also has an M.2 slot for WiFi, as well as both HDMI and DP...supports two simultaneous displays out of the box.

Why would you use the M.2 for wifi and not a super fast disk in that slot? USB bus should be able to handle wifi just fine.

Because it's a M.2 E slot, which has no SATA. Maybe there are SSD drives now that can use PCI and fit in the smaller E slots? There didn't used to be. If so, great...

just no SATA? or no SATA and no NVMe?

M.2 E has PCIe ×2, USB 2.0, I2C, SDIO, UART and PCM.

As far as I know, NVMe drives want 4 PCIe lanes.

There are recent low-cost NVMe drives that do 2x, I believe.

If a PCIe device doesn't fall back to the available channels that is most likely a bug. An NVMe from a reputable manufacturer should work in this slot.

I really don't think it's that easy. Find a picture of an m2 drive has a connector that would fit an "M.2 E" slot. The connector would have to look like this: https://www.addonics.com/products/diagrams/M2-E-KEY-WIFI-CAR...

I don't see any.

The development kit specifically lists NVMe as an option

https://www.nvidia.com/en-gb/autonomous-machines/embedded-sy...

Tried "search in page" for NVMe, no luck. Not sure what you're seeing.

Weird. I opened it in reader mode. Doesn't seem to be on the normal page. It's a screenshot, maybe a draft?

https://www.nvidia.com/content/dam/en-zz/Solutions/pattern-l...

The "PCIe x 16" probably means that goes with one of the previous, much more expensive, Jetson boards.

This $99 Jetson appears to have no direct support for any type of M.2 drive. Maybe some M.2E PCIe to SATA adapter board, if such a thing exists.

Cool thanks for all the info. I just assumed the M.2 slots were all standardized. Didn't realize they were different.

A more practical solution is to train your network on other beefy CPU/GPU/TPUs and convert+run that on the Nano using TensorRT. TensorRT supports import of models in Tensorflow (UFF) and ONNX formats.

Reply on news.ycombinator.com