Hacker News Reader

You missed one of the most major issues:

Most of the vendors know they have serious issues here, but hide them. Literally. You never notice because they don't use open source compilers (even for things like CUDA, let alone shader compilers), so it's not obviously more than "just a bug" until it happens to you continuously. Instead, they pretty much never have to fix the bug until someone notices, and then they hack it some more and move on, instead of fixing underlying issues in their structurization/etc passes.

Most vendors i've talked to can't even tell me what control flow breaks their compiler (again, doesn't matter if we are talking shaders, cuda, you name it. it's all broke), they know plenty does, but are fairly ¯\_(ツ)_/¯ about doing more than working around whatever bug they get given.

Meanwhile, over in open source clang/llvm world, we can basically fuzz test CFG's, etc for CUDA.

The death of some of these compilers can't come fast enough.

"You never notice because they don't use open source compilers (even for things like CUDA, let alone shader compilers)"

Actually AMD recently completely open sourced their entire GPGPU compute stack. See https://github.com/RadeonOpenCompute

Compiler, assembler (LLVM based), linker, driver, etc. Everything is open to my knowledge.

Actually, we wrote the CUDA support in clang :)

and yes, AMD has done this because they have nothing to lose anymore :)

Even worse some of the mobile GPU vendors, coming from the hardware/silicon industry, sometimes provide fixes to their bugs only when a customer asks for it, leaving the fix out of the master repository for fear of providing evidence to potential litigators and IP to competitors.

In my experience, the situation is a little better for AMD and NVIDIA GPUs - likely because they have already seen (relatively) heavy use for OpenCL and CUDA. The drivers are still pretty bad altogether, with some of intermittent faults and weird quirks, but compiler bugs are infrequent.

It is however my general impression that the graphics parts of GPU drivers are generally more buggy than the compute parts. Likely because the graphics stack is older and more complicated. Is this a correct assessment?

Also because the graphics parts have to implement more logic logic; generally computing users of GPUs want access to arithmetic primitives, not complicated graphics-relevant functions.

> It is however my general impression that the graphics parts of GPU drivers are generally more buggy than the compute parts.

I'm not a graphics driver developer, but I know that game engines for modern games often are actually quite buggy and often the graphics driver stack tries to work around engine bugs. It's also well-known that for AAA titles the graphics driver internally even often replaces the game's shaders completely by hand-optimized ones made by the GPU vendor. I have read from an insider that in the last years NVIdia's graphics drivers often did not validate arguments (for a little bit more performance), while the AMD's did. This was as I heard the reason why one of the last Tomb Raider game had initially problems on AMD GPUs.

Also for OpenGL drivers you have to realize that in the past Intel drivers did not really support modern OpenGL versions. So at least for Indie developers the safest way was to target OpenGL 2.1 + extensions. Though OpenGL >= 3.1 should better be initialized using a Core context many developers still use(d) a Compatibility context - this adds a lot of legacy stuff that one has to stay compatible with and thus makes the whole OpenGL implementation of Compatibility contexts very error-prone (in the driver). Luckily we now have Vulkan (though writing your engine against Vulkan is at least at the beginning more complicated (time to first triangle) than targeting OpenGL or DirectX 11.x (not DirectX 12 - DX 12 is a lot more like Vulkan)).

In the open-source-everything world, it's a lot more common to push for and eventually make a fix where the bug is, and then tell people to update or patch. (Though it still doesn't always happen.)

In the closed-source and hardware worlds, it's a lot more common to implement a work-around wherever you are. (Alternatively, the entity paying money pressures the entity being paid to put in a workaround on their side.) Soon there are a bunch of workarounds in common use which in turn need to be worked-around. This is the world of commercial games and graphics drivers.

I used to work on a WiFi access point product, though I only touched the actual driver a few times (other team members focused there), and there was a lot of working around bugs, and working around work-arounds in other WiFi devices for other bugs in WiFi APs, in that world too. Some crazy stuff ...

With the #2 issue -- it would be nice if the drivers were distributed as a drivers only option rather than a mega-advertising option. It looks like AMD does this now with a 55MB version, but Nvidia still only has the ridiculous "experience" application in place of a simple driver package (300+MB).

I understand your opinion as a consumer but it's not a real problem (minor annoyance at most) compared to millions of consumer devices with outdated drivers (with possible security problems).

The "experience" thing is optional afaik.

Let's meet in the middle and do it like Apple?

Also, as a consumer the "experience" seems to matter a lot more to me than ... whatever the other side of this argument is ... the hardship for programmers?

And since you're bringing up security: if there are security holes in GPU driver installations, I'd bet eliminating all the crapware would do a lot more for security than any number of updates could ever do.

The worst place for security bugs in drivers is mobile, for the normal mobile reason (try getting security updates for any OS component!), and there's no crapware there.

Yes, some kind of meet in the middle solution would be nice. For example, Microsoft's official updater keeping the drivers up to date. And by up to date I mean the latest from the GPU vendor and not something a bit older but certified by MS.

Getting rid of 3rd party installers and updaters should be a priority of Microsoft's anyway.

> if there are security holes in GPU driver installations, I'd bet eliminating all the crapware would do a lot more for security than any number of updates could ever do.

The "crapware" is just some unprivileged user space processes. The driver has kernel mode components, and security issues there could cause crashes, snooping memory of other processes or even kernel space remote code execution.

Not that I'm defending bundled (or optional) crapware, but I don't think that's a security problem.

> Let's meet in the middle and do it like Apple?

Apple's OpenGL drivers (on desktop at least) are the worst in the industry, and that is not an exaggeration. I have found so many cases in which you can make shaders read random memory ([1] for example) that I'm no longer surprised.

[1]: https://github.com/servo/webrender/commit/96ac2f13b9b73dcfa0...

Just took an applied parallel programming class at a top 5 engineering school. The untapped potential for GPUs really is crazy. If GPUs could be used for common tasks able to be parallelized, then less work for the CPU and way quicker computations. I mean we made a convolutional neural network for handwriting recognition which took ~30 minutes to run on a CPU, ran it parallel in 400 milliseconds on GPUs. Once CPUs start leveling out due to the transistor-heat issue, I think GPUs might start taking on a bigger role, they simply need more support. So many people are all about the buzzwords in the tech world that no one wants to turn the attention something like parallelization, but hopefully someday soon

Going from 30 min on CPU to 0.4 sec on GPU can mean only one thing - you had a really crappy CPU code.

A well written C code should run about 20-50 times slower than a well written CUDA code on the latest GPUs, and that's for a single CPU core. This was true with my hand-written implementations, as well as when using libraries (Numpy vs CuDNN).

Going further, power for power on the current generation, for SGEMM, i'd expect GPUs to beat CPUs by maybe 5x (6700k has a theoretical 512 gflops at 91W, while xeons can come a bit shy of 1.5 tflops at 145w)

Which is still pretty major, but nowhere near the wild claims that used to be common in academic papers. It's outdated, but [1] is still pretty good reading.

[1] http://sbel.wisc.edu/Courses/ME964/Literature/LeeDebunkGPU20...

We implemented the LeNet-5 convolutional neural network, and yes when run sequentially it really does take that long. The sequential code that took that long was also written by professors who worked at NVIDIA, it was written as well as it could be sequentially. The project was a competition to see how fast we could have it go in parallel so we used a ton of parallelization techniques and tricks like streaming to make it go at .4 sec. Think again before you call someones code "crappy"

I'll second p1esk calling the CPU version barely optimized based solely on the numbers you're providing. Unless you're comparing a D410 against a Titan X?

It was run on a tesla k80 cluster, so pretty high end. But the reason we were able to get such acceleration is because we wrote the the low-end CUDA code, using as much register memory and shared memory as possible, while using good streaming techniques (which really accelerates it), as well as matrix multiplication techniques faster than SGEMM. The batch size was also huge that the code was run on. Since we wrote the low-end CUDA code we were also able to prevent control-divergence as much as possible by using knowledge on warps and how dram bursts occur. We weren't using any API like openACC or anything for help, we wrote CUDA code with some pretty good optimizations. The numbers are real, and a lot of the optimizations come from understanding how the nested loops in the LeNet-5 CNN work together. All I'm saying is this shows how well GPUs can speed things up when writing efficient GPU code.

I'll add my vote to the side saying "poorly optimized CPU code". This doesn't mean that the code is "crappy", but if you are more than 1000x faster on the GPU than CPU, there is almost certainly the potential for improved performance on the CPU. Optimization is hard, and depending on their area of focus, professors who worked at NVIDIA may not be in the best position to get top performance out of modern Intel CPU's.

I'd be betting that someone with low level optimization expertise (comparable to what you appear to have done with the GPU) could get at least another 10x out of the CPU version. You are completely right that GPU's have the potential for great increases over the current normal, but there's also (typically) room for large improvement on modern CPU's as well.

It's actually about 75x faster, not 1000x. I mean there's a difference between parallelizing basic code vs something like the LeNet-5. The sequential code was pretty standard. I mean I'm sure the CPU code could've been optimized more, I'm not trying to argue, I just thought it was interesting to see how fast you can make some CUDA code go

Wait, did you make a typo, and it was 30 seconds for the CPU run time, not 30 minutes? If so, then yes, that's more realistic.

Oh wow. Now that's embarrassing. My bad for stirring things up with that

You can actually get some gain in CPU performance by writing in OpenCL. That's because OpenCL code is meant to be easily parallelisable and consumed by wide SIMD units, so Intel's compiler can do a lot more autovectorising than for C code.

I just did a quick test for Lenet-5, using Theano + CuDNN 5, Xeon E5-1620v2 3.7GHz, Maxwell Titan X GPU:

Two conv layers (6 and 12 feature maps, 5x5 filters), one fully connected layer (120 neurons), activation=Tanh, pooltype: average (excluding padding), cost=Negative Log Likelihood.

Learning rate=0.20, minibatch size=100, dropout = 0.0, L2 lambda=0.0, momentum=0.0, initialization: normal

Training for 10 epochs:

CPU: 172 sec, GPU: 14 sec.

When decreasing batch size to 20 images, the numbers are: CPU: 318 sec, GPU: 38 sec.

So yeah, the CPU code your professors wrote was really crappy.

> Think again before you call someones code "crappy"

Lets see it then. Most cpu programs have enormous amounts of fat left in them from cache misses and poor or no use of SIMD.

What causes #1? Why's OpenGL so troublesome and weird for each device?

Easy just think how browsers render HTML/CSS/JavaScript, C compilers specific interpretation of ANSI C, UNIX variations from what POSIX leaves to each implementation, and many other implementation vs standards variations.

Just like them OpenGL is a text standard describing how a 3D API is supposed to behave.

It happens that what those papers state and what each team of developers at every card manufacturer understand is not always the same.

Then there is the set of card specific extensions, for every nice feature they want to sell on their graphics card, but yet to be adopted by OpenGL paper standard.

There are of course certification tests available, but they are costly and don't cover 100% of the API usage anyway.

So programming OpenGL, happens to be like trying to write web applications, while trying to make the code portable and bug free (with workarounds) across all graphic cards out there.

> So programming OpenGL, happens to be like trying to write web applications, while trying to make the code portable and bug free (with workarounds) across all graphic cards out there.

And it's much harder to test because you have to find the outdated graphic card, with the outdated driver, and have a machine where you can plug the offending card.

One reason is that shaders are stored as text and require each driver vendor to write parsers for them and not everyone implements them in the same way.

Further, the drivers have to be more permissive than necessary. It's not an option for a GPU vendor to ship a driver update that would break Doom/Quake/Dota/etc.

Ideally games would ship only shaders that are strictly valid GLSL but that's not quite the case.

If you're developing GL, do everyone a favor and start using the official GLSL validator in your build scripts.

Compounding issues #1 and #2 you mention is that when the HPs, Lenovos, et al do ship updates they're usually a few months behind current at best. I've seen this for Intel, AMD, and NVIDIA. What's worse is if you run the "generic" Intel GPU driver installer, if it detects an OEM's driver on the system it will refuse to install.