Nvidia Deep Learning Accelerator (NVDLA): free open inference accelerator (2017) | Hacker News Reader

Nvidia Deep Learning Accelerator (NVDLA): free open inference accelerator (2017)

nvdla.org

62 pointsmy12325 comments

The cynic in me is confused. Nvidia isn’t exactly known for making lots of things open source. What’s their angle here? Is the idea to take steam out of projects like TPU by making viable, open-source alternatives?

Seems pretty straight forward to me. Their maximum profit comes from selling more GPUs, instead of trying to sell a piece of software that has open source competition.

It's not like their sales people are having a hard time pushing the hardware at the moment. (In case someone hasn't noticed: All GPUs have been sold out for months.)

Sure, but they have to think ahead. GPUs make sense for ethereum, but not bitcoin (which moved from GPUs to FPGAs to ASICs). Ethereum is moving from proof of work to proof of stake. I expect the used market to flood with used mining GPUs once that happens.

Commoditize your complement. Nvidia wants to sell neural net training chips. Neural net inference chips are a complement.

> Nvidia isn’t exactly known for making lots of things open source.

It is true nvidia drivers are not open source, but this github page says nvidia has 248 repositories: https://github.com/NVIDIA

I'm not sure why the link was submitted today. The repo says it is 3-4 years old.

I understand why Nvidia gets a lot of heat from the Linux community regarding the closed source nature of their GPU drivers. That said, I use them on all of my Linux machines and not being a FOSS purist I don't take issue with it. Additionally, most of the complaints come from the Linux desktop community. You rarely see the ML community at large expressing any concern. For the most part they're just trying to get work done and the overall CUDA ecosystem is tremendously useful in that regard.

Very usable desktop experience, CUDA just works, the rest of the software ecosystem makes anything else pale by comparison (see the Pytorch ROCm discussion that made the HN front page today). Every few months I take another look at ROCm and frankly don't understand how anyone can take it seriously.

In the FOSS/Linux community the discussion around the closed source nature of desktop drivers seriously discounts the amount of FOSS work Nvidia has done.

My perception is that Nvidia in particular gets a lot of heat because they have bad/buggy drivers which integrate poorly with existing package managers/distros, not just closed source drivers. Anecdotally I have at least one issue on my home desktop which is suspect is caused by Nvidia drivers and poor display hotswap on a work device which is definetly caused by Nvidia.

That's fair but every time the issue is raised on HN, for example, there are anecdotal reports all over the place. Given the ridiculous number of variables I will believe virtually anything to be true for that specific person, configuration, use case, etc.

I understand the support for AMD's in kernel drivers (and Mesa), rooting for the underdog, etc but anyone who is being honest will acknowledge there are plenty of issues with their drivers and various configurations as well.

Assuming it's six of one, half dozen of the other the fact that Nvidia drivers also bring the entire CUDA ecosystem to any GPU made in the last five years (or more) really tips the scales (overall) for the justification of an Nvidia hardware purchase (all other things being equal).

NVIDIA has a lot of open source. https://github.com/NVIDIA

NVIDIA's business is to sell GPUs. This helps.

I'm actually kind of confused here; NVDLA has been around for a while. What is new or interesting here that changes anything? I did a cursory look to see if there's any new announcements, but - even things like the roadmap on the site show a 2017 roadmap, and the last commits to the hw and sw repos are 2019. Perhaps this needs an edit to say it's from... 2017? Or is it 2019?

I only learned about NVDLA recently, from reading the specs of the BeagleV board that another poster has linked to.

Should definitely say (2017). Looking at the Wayback Machine from November 2017, I don't notice any differences from the current page.

Added. Thanks!

Cool beans. :-)

A componentized DL inference hardware library + compiler suite + inference runtime. Overall already appears a solid foundation for a production system ATM. Another example of nVidia's unchallenged prowess in DL software ecosystem.

The hardware design library could be seen as a prominent example of "software defined hardware". Specifically, the compnentized hardware library dissects DL inference into 4 high-level operations: convolution, pooling, activation, normalization. Hardware description code is provided for these 4 operations. Allowing practitioners to tailor their chip design to their application. For instance, one could remove an operation from the chip, and adjust the size and throughput of any one operation.

Software: compiler suite, and runtime are provided for building and running actual DL models. Compiler suite also operate based on the 4 operations, and produce an IR format that models around the operations; and then optimized towards the target chip. Runtime include kernel module and user space driver to handle the setup of the chip runtime and accept user request of loading model and inference requests.

2 questions I did not find explicit answer:

* Is there an inference-aware training capability?

* Does the compiler suite also consider the actual hardware spec during its optimization?

NVDLA [1] targets IoT devices:

> NVDLA is available for product development as part of NVIDIA's Jetson Xavier NX, a small circuit board in a form factor about the size of a credit card which includes a 6-core ARMv8.2 64-bit CPU, an integrated 384-core Volta GPU with 48 Tensor Cores, and dual NVDLA "engines", as described in their own press release. NVIDIA claims the product will deliver 14 TOPS (tera operations per second) of compute under 10 W, but most of this likely comes from the GPU cores. Applications broadly include edge computing inference engines, including object recognition for autonomous driving.

> NVIDIA's involvement with open hardware includes the use of RISC-V processors as part of their GPU product line-up.

[1] https://en.wikipedia.org/wiki/NVDLA

The BeagleV (https://beagleboard.org/beaglev) also has an NVDLA in the SoC. (1 core w/ 2048 MACs at 800MHz)

In the license:

> “Other NVDLA Rights” includes copyright, design right (whether registered or unregistered), semiconductor topography (mask work) rights, and database rights to the NVDLA Specification and any Derivative Work. For the avoidance of doubt, Other NVDLA Rights does not include patents or trademarks.

> Subject to the terms and conditions of this License, NVIDIA and each Contributor hereby grant to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable license under the Other NVDLA Rights to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the NVDLA Specification and such Derivative Works, and to commercially exploit any mask works included in the NVDLA Specification or such Derivative Works

Why would they grant you permissions to "semiconductor topography" rights?

Because of this I imagine? https://en.wikipedia.org/wiki/Semiconductor_Chip_Protection_...

Right, but why would Nvidia want to give you the rights to their silicon designs?

Is there any information on how to source components outside Nvidias Jetson ecosystem? It would be interesting to be able to buy such chip individually for e.g. embedded projects but there seem no vendors listed. Considering the DLA project is not particularly new I wonder if it will remain a niche appliance on Nvidias boards.

This is 2+ years old, and the Github has seen zero updates. Seems like a dead-end effort to me.

The Microsoft buying Github isn't an encouragement to contribute for some people knowing what they know of previous Microsoft leadership.

> a free and open architecture that promotes a standard way to design deep learning inference accelerators.

Is it really open and standard, i.e. not using anything like CUDA?

Reply on news.ycombinator.com