How Jensen Huang's Nvidia is powering the A.I. revolution
newyorker.com
newyorker.com
Hidesight though
Tricky question to answer.
When AI moves further away from academia NVIDIA will have less of a grip.
Proprietary, hardware specific APIs never stand the test of time. Ask 3Dfx.
Either CUDA will open up, if it is to survive or open API use will spread.
Weirdly, NVIDIA hardware only outperforms competitors on its own API. When you compare NVIDIA on a level playing field, they aren't the clear winners. Nobody is right now.
I suspect the battle ground for AI will be accuracy rather than speed in the medium term and on paper AMD could win there...purely because they aren't shy about over speccing the RAM in their kit and certain price points.
For me, I want to run the largest models I can with the least amount of quantization for the best bang for the buck...and AMD is right there as soon as people start picking up APIs outside of CUDA.
Until university labs get people working in open frameworks and not CUDA, every student joining the industry will default to NVIDIA GPUs until they're forced otherwise. The few people I've managed to convert have been forced by supply constraints, not any desire to innovate or save themselves money. As long as NVIDIA can keep the market satiated with a critical mass of compute, they'll sit on their throne for a long ol' while.
Where I work, we've made it a principle to stay OpenCL-compatible even while going with NVIDIA due to their better-performing GPUs. I even go as far as writing kernels that can be compiled as either CUDA C++ or OpenCL-C, with a bit of duct-tape adapter headers:
https://github.com/eyalroz/gpu-kernel-runner/blob/main/kerne...
https://github.com/eyalroz/gpu-kernel-runner/blob/main/kerne...
of course, if you're working with higher-level frameworks then it's more difficult, and you depend on whether or not they provided different backends. So, no thrust for AMD GPUs, for example, but pytorch and TensorFlow do let you use them.
The problem is that when a company has done serious capital investment to advance a market, anyone who invested equivalently wouldn't reap the same rewards - competition would just eat away each company's profits so no one will challenge that.
They are supporting it seriously now. It is being actively developed and improved.
I don't really think so, at least not anytime soon while the hardware functionality continues to evolve so much, and while they seem to be concentrating on the high end devices/architecture rather than low-end stuff.
I've been more or less exclusively writing CUDA for the past decade in the AI/ML space (though have spent some time with OpenCL, Vulkan and other things along the way too). What a GPU is or should be I don't think has reached an evolutionary end yet. CUDA also is not a static thing, and it has co-evolved with the hardware, not being locked into some static industry standard with a boatload of annoying glExtWhatever dangling off of it. Over the past decade or so, Nvidia has introduced new ways that the register file can be used (Kepler shuffles), changed the memory model of GPUs and the warp execution model (to avoid deadlock/starvation by breaking the lockstep behavior somewhat), slowly changing the grid/CTA model (cf. cooperative groups, CTA clusters), adding more asynchronous components to the host APIs and the hardware (async DMAs), and has constantly changed the underlying instruction set, all of which leaks into CUDA in some way.
CUDA won't die if Open APIs take over AI inferencing operations. It's still used and applied in so many niche industries that it can only be "replaced" in industries like AI where companies invest in moving digital mountains. Stuff like Microsoft's ONNX project will go a long way towards making CUDA unneccesary for AI acceleration, but it won't ever kill the demand for CUDA.
Just look at how lethargic the industry's response has been in the wake of AI, and look at how other companies like AMD and Apple abandoned OpenCL before it was ready. Now Apple is banking on CoreML as an integration feature and AMD is segmenting their consumer and server hardware like crazy.
> Weirdly, NVIDIA hardware only outperforms competitors on its own API. When you compare NVIDIA on a level playing field, they aren't the clear winners.
That does not reflect any of the benchmarks I've seen at all, unless by "level playing field" you mean comparing old Nvidia chips to modern AMD ones. The only systems comparable to the DGX pods Nvidia sells is Apple's hardware, which lacks the networking and OS support to be competitive server side.
AMD is an amazing company for being open and transparent with their approach, but nice guys always finish last. This is a race between the highest-density TSMC customers, which means it's Apple and Nvidia laughing their respective paths to the bank.
Is it known, why consumer cards do not have fp64:fp32 performance at about 1:2?
3DFX lost for other reasons. DirectX is doing fine despite having had open competition for decades.
they have the best hardware for it. Their vendor lock in is just CUDA, but that’s not really much lock in.
You can train models on AMD cards too, but why would you when nvidia cards are just better for it atm.
Nothing besides $ prevented their competitors from developing their own GPUs and software stack comparable to CUDA.
There's so much opportunity for custom silicon to massively improve compute-per-watt -- for both for training and inference. Nvidia got lucky with their high-memory GPUs being early to this space, but the usefulness of that architecture has already peaked.
I don't really see NVIDIA's dominance changing (other than whatever AMD has in the pipeline) Unless VC's change their risk appetite and take interest in HW startup funding which seems unlikely considering the avg returns and success rate of HW startups.
Eg, Google (TPU), Apple (NPU), Amazon, Telsa (Dojo), Microsoft/OpenAI, etc.
This part of the article bothered me. Why was diverse placed in quotation marks? Aren’t South Asians and East Asians considered minorities?
Also South Asians are a very diverse group in it itself. As are East Asians. Many different languages and cultures….
So why was diverse in quotation marks? Why did the author say “sort of”???
"Employee demographics are “diverse,” sort of—I would guess, based on a visual survey of the cafeteria at lunchtime, that about a third of the staff is South Asian, a third is East Asian, and a third is white. The workers are overwhelmingly male."
"Overwhelmingly male" is an obvious counterpoint the author is making to the ethnicity distribution and by itself explains the quotation marks.
Everyone is free to read in additional subtext but your question can be answered solely by not dropping the second sentence of the author's.
Europeans and their diaspora (aka White people) are also very diverse in the sense of many languages, many cultures, many phenotypes, varied histories, etc.
But we all know what capital-D "Diversity" means here, and it is not this.
What does whether someone is a White or Asian got to do with his competency on the job?
If you ever wondered if you had what it took to be a NYer writer, consider if you could have provoked & recorded this vignette.
I asked Jeff Williams, the senior vice-president, if the Apple Watch seemed more purely Ive’s than previous company products. After a silence of twenty-five seconds, during which Apple made fifty thousand dollars in profit, he said, “Yes.”
From our perspective the lives of horses look pretty horrible, particularly their lives in service to us, but I don't think it would be very comforting to have an AI explain to me that humans aren't fully aware of the world in the way that greater intelligences are, that because of the mental issues in humanity which cause them to do annoying things like bombing data centres or cutting fibre optics cables the human population needs to be significantly resized to a more sustainable number, or anything else a human-equivalent that actually cared might explain to a horse-equivalent that could actually listen.
The point is that horses don't control their own destiny, and they aren't even important enough for us to particularly care about said destiny in anything other than vagaries. Horses that interfere with human society in some way get put down. I don't want to be put down or have no say in the future, so we shouldn't attempt to build something which has a reasonable likelihood of being as much smarter than us as we are smarter than horses.
This will come from the capitalist system, businesses will be driving the obsolescence of humans, not governments.
Once the owners of the means of productions don't need humans in their workforce, what value to they bring?
It will be much harder to "revolt" against capitalism/businesses than a central government.
And the biggest danger here, is it means the end of the capitalist system as the best way to increase living standards. If capitalism doesn't work as a means of redistribution anymore, because humans have been devalued, what economic model can replace it and be as successful?
We're re-entering an era where economic systems are all going to favor a few classes, and impoverish the rest.
what's the point of owning the factories if there is no one to buy the crap you produce? they need us more than we need them, at least until they have their own spaceships with AI and robots to fluff their balls for them.
You produce things you yourself want to consume. Why produce things for others if they have nothing you need?
The only reason there is a need to sell to workers today is so you can pay them a salary to work in your factories, and then take a portion of that for yourself. When you don't need workers you don't need to sell to workers, you just take the entire output for yourself.
Edit: If you can't imagine how someone could consume that much, then think of a golf enthusiast so he tells his AI factories and builders to prepare new golf courses at the same rate he can finish them, so he can play gold all day long and never replay a course.
Basically production will become almost 100% luxury focused instead of median consumer focused like it is today.
There is no legal reason why AIs couldn't be governed under the same principle.
You shouldn't just be worried of your own government here.
And the other issue is, why would the post-revolt government/businesses abandon AI/robotics?
The truth is, this won't happen over night. People will have less work over time, be paid less and less, the gap between classes will grow bigger, etc.
I guess another way to put it, take a country like India with lots of unemployment and high corruption, I don't think if they had guns they'd all revolt. You have to believe there's a point to revolting, an alternative that's better and worth the risk.
How much political power do you think the purveyors of AI wield?
> As long as we live in a democracy people will not become horses.
It’s estimated that 2.3 billion people—about 29% of the global population—lived in a democracy in 2021 (https://www.visualcapitalist.com/cp/how-many-people-live-in-...)
And if you live in a democracy today, as you yourself state, it is not guaranteed to remain that way.
Britain experienced an economic collapse in the early 5th century AD after the Roman Empire left. Economic activity and urban life decided, and buildings were abandoned. Their supply chain was suddenly much more limited, and localized to specific areas, whereas before, the Roman Empire had allowed them to trade with other countries more. Their standard of living didn't really change for the next few centuries. I won't compare beyond that.
For example, in Korea during the Joseon/Chosŏn period, the status of women gradually declined due to Neo-Confucianist ideology. Women stopped inheriting property in the seventeenth century. They lost the right to intiate divorce, while men could still intiate divorce under seven grounds (disobedience to parents-in-law, failure to bear a son, adultery, jealousy, hereditary disease, larceny, and talkativeness). Widows lost the right to remarry, and were seen as inconvenient for the family. Women were forbidden from playing games, partying outdoors, and riding horses.
The decline of women's rights happened in other societies, too; I just happened to have a book about Korean history on my desk.
North Korea, after the Korean war, their living conditions got way way worse than when under Japanese occupation, even though their military has improved.
Afghanistan, since it's been overtaken by the Taliban, even though they are mightier from a military standpoint, the living conditions have become worst.
The Soviet Union post WWII, they came out of it as a military superpower, but the damages from the war meant their living conditions were way worse.
I can’t tell if you’re being very clever here.
At the risk of being whooshed, I’m going to say that the common man has a lot more in common with horses than billionaires.
This is handwringing over the decline of paper or pencils after computers came about when paper didn't magically come about by itself. Horses were not wild animals in North America at any point in modern history and barely in other continents. They were purely bred for industry, travel, and hobby. When industry and travel no longer became a thing we stopped producing them: a clear 1-to-1 correlation.
There's already been reductions in human populations without AI from leisure, abundance, culture, etc, - a very very different causality than horses.
You make a very compelling argument. It made me think from another angle. Consider that there's evolutionary pressure on humans to want to procreate. So in a certain sense, we are conceived to serve the gene and to survive we have to labor.
And when you're done doing whatever it is that is, do you go home to your home, your stable, if you like, to sleep and eat?
Are you truly so different from a horse in the scheme of things just because you spend a portion of your free time staring at a mobile phone?
[^1] That could be a good thing if our AIs are destined to replace us...it means it will take longer, because the AIs will have to pay very high taxes to NVIDIA[^2].
[^2] Or to the AIs in control of NVIDIA.
Back in the early 90s there were countless 3D chip startups, probably more than 35 (I was at two of them during those years) and they fell one by one until essentially only Nvidia and ATI remained. Ask the other 33 CEOs if they think Jensen was just lucky.
Jensen believed that GPGPU could be a new market and spent heavily to push the idea and to make it real. Significant design work and silicon area was spent adding GPGPU features to chips before there was any software or any market for it. He invested in building compiler teams and writing libraries. He invested millions each year hosting a GPGPU conference and driving the vision. No, he didn't write those libraries or design the silicon, but it was his vision and willingness to invest in it that deserves respect. He isn't some schlub that simply got lucky one time. He has been manufacturing "luck" for 30 years.