Hacker News Reader

Comment by lucasyvas | Hacker News Reader

I'm going to get raked across the coals for this but I think it's time for the Linux developers to finally put a stop to NVIDIA's antics. Force them to release an open source driver - they will comply, otherwise their AI conquest dreams will be over.

Start playing hardball with these vendors and either put an end to proprietary blobs or make it a living hell to achieve it with no compatibility guarantees. You're playing in the GPL sandbox so do things their way or get lost. NVIDIA should stick to Windows or another Unix if they don't want to play nice.

I'm betting people would rather not take on the operational burden of a different OS in their stack long term and would prefer a vendor that works with Linux well.

View on HN

There is still no replacement for CUDA. ROCm is a compatibility joke, Vulkan is too insanely low-level, OpenCL runs way too slow. Compute shaders are technically fine but bring flashbacks to 2006. Metal is a good alternative but only offered on systems without discrete GPUs! so we’re running CUDA for the next five years. In a conflict between Linux and Nvidia, in the HPC or AI spaces, Nvidia wins.

>In a conflict between Linux and Nvidia, in the HPC or AI spaces, Nvidia wins.

I HIGHLY doubt this. Nvidia has decades of investment in Linux internally. They aren't going to convert their DGX to Windows, or their dozens of internally utilized supercomputers. Sure, in a small window end-users are going to make a LOT of noise about Linux "breaking" - but unless Nvidia is planning on forking the kernel and fully maintaining their own version of Linux, they'll cave.

> ... but unless Nvidia is planning on forking the kernel and fully maintaining their own version of Linux, they'll cave.

Why wouldn't they? It would only be a few million dollars a year to do so, at most.

Their customers are not going to run an Nvidia fork, because the vast majority of applications only compile on very specific Linux configurations to begin with. This would effectively cede the market to CPUs or AMD.

No, it would be a lot easier to modify their applications then wait for AMD to create CUDA 2.0.

Yeah, they totally would.

> but unless Nvidia is planning on forking the kernel and fully maintaining their own version of Linux

https://docs.nvidia.com/dgx/dgx-os-6-user-guide/introduction...

That isn't a permanent fork away from the Linux kernel. It's basically them repackaging Ubuntu LTS, customized with some of the binaries packaged in.

Permanently forking the Linux kernel us a VERY different beast then taking Ubuntu, pre-installing Nvidia drivers, and then re-baselining everytime Ubuntu releases a new LTS.

Entirely depends on how extensive the patches are. FAANGS will all have their own kernels with custom patches, some of which will find their way back into the distros, and some which are kept in-house for custom reasons. It isn't necessarily that burdensome.

And I was involved in doing this at Amazon (although a long time ago now).

NVidia also doesn't really need to fundamentally fork the linux kernel to do this since drivers are already a pretty hard API boundary layer. They won't need to touch the TCP/IP stack, scheduler, etc. Their version of a "fork" would be "add back the API which is being removed here".

Lol, no they won't. The cost of developing their software stack is massive. They'd either resort to more technical hacks like running more of the driver in userspace, or if push comes to shove just develop a Linux compatible kernel with a cleanroom codebase. Kernels aren't that hard to develop for a company like NVIDIA especially if you only care about supporting a limited set of hardware (like DGX servers).

Linux wins hands down, it always has.

Nvidia would win this fight. Linus and co taking a hard stance would result in the industry bifurcating on Linux. AI just has so much momentum and money being poured into it.

webgpu will eventually fill this role right?

Isn't that exactly what has already happened? Nvidia has released a fully open-source kernel side driver for their current hardware, probably because of pressure from the hyperscalers that don't want to run closed-source code in their kernels. The new driver still needs to be integrated into the upstream kernel so this news is still about the current legally dubious one.

Unfortunately there's not really any competition in the GPU space for AI use. AMD hasn't stepped up, and Intel is barely getting its footing for basic gaming functionality. Nvidia has an established monopoly on the GPGPU space, and forcing their hand, and them withdrawing from Linux compatibility, would result in everyone that needs Nvidia to ditch Linux. At least until an alternative becomes available, and then those customers would ditch Nvidia for screwing them over once already.

This is where WSL pulls its mask off. Windows will become (even more so) the platform of choice by being a "more compatible" version of Linux than Linux. And AI developers don't seem to have much affinity for open-source. Why would they care about being stuck on Microsoft and Nvidia's proprietary platforms when they're building their own proprietary models?

Great point. WSL has always been a wolf in sheeps clothing, in my eyes.

I'm not saying it's not useful. It certainly fills a role people clearly wanted filled. But you'd be naive to think Microsoft isn't taking on an "embrace, extend, extinguish" mentality. Again.

Also, I'm sure AI devs would be all for making it more difficult to run models on the desktop so you'd be forced to use a cloud service. They could even keep using a non-GPL patched Linux on their servers because it's not like they're distributing anything.

One word: Apple.

With Apple's significant work from hardware on up with AI there's a very near future where many AI applications will train models on a Mac[0] directly in XCode[1], bundle it with the iOS/Mac OS application (and browsers), and run inference directly on the device. Apple will also be deploying more and more of their own foundational models shipped in the hardware/OS. Most user AI applications will be as simple as "call this coreml API". With the links provided it could be said we're already there and they've been laying the groundwork for years.

This hits Nvidia and big GPU cloud providers two fold:

1) Training. We already see with the unified memory architecture of Apple Silicon that Apple can provide accelerated training with RAM capable of training/finetuning very large models[2]. 800GB/s bandwidth memory (192 GB!!!) is very close to the RTX 4090 (1TB/s). Also note how many PCIe X16 slots the Mac Pro has... You can be certain that follow up Apple Silicon will eat closer and closer to Nvidia datacenter GPU performance/capability and with Apple's resources potentially even surpass it for all but things like training massive LLM models from scratch a la Google, Meta, OpenAI, etc.

2) Inference. Why pay AWS or whoever to run your inference when users have already spent their money on hardware that does it faster (because network latency) and more privately on the devices they already have? iOS 17 already does some AI tuning (FINALLY) on device with auto-correct - expect to see this show up everywhere in iOS and iOS applications.

I don't make many predictions but this one is obvious. I'm convinced the next generations of Apple Silicon and the Apple software ecosystem, developer experience, etc will be the first very real threat to Nvidia. When this really gets going Intel and AMD will be permanently left at the starting line (where they are now) for all but very bespoke use cases and deployments like Frontier[3].

For people who take issue with MacOS look at the progress of Asahi...

Then, once Google/Android catches up things will get REALLY interesting. This is the future and the current grab for Nvidia GPUs and clouds will eventually look like crypto mining - a few great years to be capitalized on when the getting was good.

[0] - https://coremltools.readme.io/docs

[1] - https://coremltools.readme.io/docs/xcode-model-preview-types

[2] - https://www.apple.com/mac-pro/

[3] - https://www.amd.com/en/products/frontier

> 800GB/s bandwidth memory (192 GB!!!) is very close to the RTX 4090

40xx is not the competition. DGX H100 has 2TB of memory, H100 NVL has 2x4TB bandwidth and are selling as fast as they can be made at any price they ask.

I quoted the 4090 because this is the Mac Pro on the second gen of Apple Silicon in a desktop/workstation. In a few years Apple has nearly caught up to the flagship desktop GPU from Nvidia - who has been at this former niche for 15 years. 192GB is already enough RAM to finetune very large LLMs and we will be seeing more memory from Apple down the road.

192GB is 4 RTX A6000s or 2 A100/H110s ($20k just in GPUs)... A completely maxed out Mac Pro (with wheels and fancy mouse) is under $13k. Oh yeah and you can actually buy them.

The M3, M4, and beyond will have TFLOPS and RAM jumping by leaps and bounds. CoreML will start chipping at CUDA developer mindshare from academia on up - you can already run LLMs at more than reasonable speed on a $2000 Mac Book you already have laying around. Expect to see Mac Pros showing up in AI labs at universities and course work where every student just buys a Mac. Of course at this level it's mostly higher level frameworks anyway so transferrable knowledge to CUDA powered whatever.

As I noted, the big stuff from Nvidia will own the very high end (especially for training) for the foreseeable future and there's plenty of money to be made there as you noted.

But Apple is going to take big chunks out of Nvidia on the hardware side and a host of startups from the software/SaaS side. For hardware less demand on the low-mid side and less demand on the inference hosting side even for the big players.

For software leveraging foundational Apple models, dev tools, etc will make it so that any app developer can drop data in Xcode to tune/build a model for their app. Apps will be able to further finetune with user data on device.

See the links I provided - this is here (in beta) today.

Have you seen Nvidia's valuation lately? They are printing money because everyone wants it. I hear you but that is a utopian pipe dream that will never see the light of day. This is the gold rush.

> Have you seen Nvidia's valuation lately

That's exactly why it's time to strike. They don't want to cause any waves that would stop that momentum. This is the perfect time to intimidate them into falling into line just to avoid a public stink.

There is no way that the Linux people are going to intimidate NVidia. This is just not going to happen, sorry. There won't be a public stink. All that will happen is that Linux will lose market share and that people will stop seeing Linux as a viable alternative because they are documented as picking fights with industry giants they can't possibly hope to win.

Intimidate them? My brother, they are valued in the trillions of dollars. Also how will you orchestrate this coup and get everyone onboard? Impossible.

This is quite honestly the Richard Stallman take. How has that gone? As admirable as it might seem - it has no legs.

nVidia's entire valuation right now is riding the AI bubble. All you have to do is convince the market that this disagreement might hurt that.

Doesn't change the fact that everyone and their brother is ordering H100's as fast as humanely possible and going all-in on CUDA. I agree there is a bubble, but the hype train is going to keep riding for some time.

One article saying "AMD playing nice with open source, nVidia not, will nVidia continue to be a leader in AI?" and they'll fold

That's a daydream. NVidia never played nice with OSS. People still use it and are doubling down on CUDA.

I personally convinced Microsoft to rename GVFS a few years ago (because same name as a GNU project) using similar tactics, just because I thought it would be funny. All it took was front page of HN, a few reddit re-posts, and Slashdot, all driving traffic to a github issue. They went from outright refusing to ever doing anything about it to completely bending over in a matter of days.

Major corporations with their toes in the open source space are _terrified_ of dev communities turning on them.

Microsoft politely changed the name of an Azure thingy that no one in the world cares about because its name conflicted with the name of a GNOME thingy that no one in the world cares about. The analogy with Nvidia's product line is imperfect.

The valuation means nothing since Linux is the more important technology by a landslide. One is the foundation of literally everything - and the other is faster at graphics and leading an emerging LLM market. Computers still work without AI - anything else in the short term is pure hype.

My MacBook Pro crushes Llama - NVIDIA hardly has an exclusive advantage.

There's no time like the present - the Linux maintainers should have told them to fuck off ages ago and they are being punished for Linus' stance now. Linus is right about a lot of things but letting this charade go on for this long was a mistake.

"NVIDIA hates everything Linux stands for" would be a hell of a slogan they would have to adopt.

And honestly to some people - who gives a shit if they're the market leader in AI chips? There will be someone else that emerges as a competitor that does play nice. What do we lose by telling NVIDIA to piss off? 2, maybe 3 years tops?

This is effectively the last chance I see for the two to divorce and be happier in the future. Otherwise they will be locked in a shitty relationship forever.

>Force them to release an open source driver - they will comply, otherwise their AI conquest dreams will be over.

Hahahaha. Absolutely nobody in the AI space gives a single shit about the GPL. Half of the big players and half of the research institutes have tried one way or another to bastardize what "open source" means.

If they have to install Nvidia-linux which is a fork of Ubuntu with very minor patches in the kernel, they will.

I'm pretty happy that things are the way they are. Linux distros (and the kernel maintainers) can't afford to pick fights with one of their major engines behind Linux adoption in academia and industry.

NVidia comes first, then the OS decision in many contexts. You pick your fights with care, whatever your personal belief in the outcome of your bets, I would not be so sure about that one myself.

Whatever. From a pragmatic perspective it's a reason to stick with Windows.

There's quite a lot of questionable telemetry in the driver, I am not sure how it would work if exposed to GDPR scrutiny like that.

This is one of the reasons people want an open source driver: so Nvidia can be called out for that undeniably (even a source-available licence would achieve that) or so OSS devs could release a forked version without if Nvidia can't be shamed into doing so.

Reply on news.ycombinator.com