Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

https://findthatmeme.com/blog/2023/01/08/image-stacks-and-ip...

The MI300 smokes the H100 yet here we are.

...assuming the recent 1.58b paper doesn't render the entire float quantization approach obsolete by then.

Wasn't there a paper from Microsoft two weeks ago or so where they trained on log₂(3) bits?

Edit: https://arxiv.org/pdf/2402.17764.pdf

I like how you specified that it's not floating point.

If they defaulted back to a menu and taskbar-based WM, it might actually be more approachable to users who are more familiar with macOS and Windows.

And that's once you've bought an expensive Tesla/Quadro GPU too.

On the other areas where NNs are used, what I can see is training models with less data is not only plausible, but very possible, esp. in image processing, probably an image carries more information than a single sentence.

I think AI falls on the spectrum with a slight bias to the latter group. Because if you can shrink a model 10% and lose a month, you'd rather have a 10% bigger model now, and reap the money^H^H^H^H^H fame.

I don't know anything about a typical web app, because I'm not a typical developer developing web apps. :)

I work on material simulations. I make processors hit their TDPs, saturate their pipelines and make them go as fast as they can. However, sometimes we come up with a formula optimization which does things 1-2% faster, which means we can save hours on a bigger computation. Utilization doesn't change, but speed does.

> I think it is more of an information problem.

It's an interesting point of view, and partially true. However, we're still wasting "space" by just adding bits to the network to make it contain more data.

There's a long way to go. "Wasteful development" is a phase and always be part of software development. The important part is not forgetting that optimization exists. Otherwise we can't sustain ourselves much with all that energy use.

Apple is in talks with Google to bring Gemini to the iPhone, and it will obviously also be on android phones. So almost every phone on earth is poised to be using Gemini in the near future, and Gemini runs entirely on Google's own custom hardware (which is at parity or better than nVidia's offerings anyway).

The move from PoW to PoS for most crypto networks in combination with bust of ‘22. NVDA slid down in value.

OpenAI debuts ChatGPT in late 2022 and now it’s suddenly bumping in price as the hype and rush for GPUs from companies of all types buys up their stock of GPUs. Demand is far outpacing the supply. Nvda can’t keep up.

Thus, share price is brittle. Competition in the GPU market is dominantly owned by Nvidia. That can change, but so far openai loves using nvidia for some reason.

Competition WILL come. Maybe it's Groq, maybe AMD, maybe Cerebras. Maybe there's a stealth startup out there. Point is, they're going to be challenged soon.

Yes, CUDA, but CUDA is maaaaaybe a few tens of billion USD deep and a few (more) years wide. When the rest of the industry saw compute as a vanity market, that was sufficient. Now, it's a matter of time before margins go to, uhhh, less than 90%.

Does that make shorting a good idea? I wouldn't count on it. The market can always remain irrational longer than you can remain solvent.

In the past year, they had a revenue of 60B $ and net income of 30B $. Absolutely amazing numbers, I agree. The year before they had a revenue of 30B $ and a net income of 4.5B $ - and it was a rather good year. What happens next of course depend of how you judge the situation - was it a peak hype demand ? Will it stabilize now ? Grow at current extraordinary rates ?

Scenario 1 - margins get back to normal due to hype going down, competition improving etc - in this case the company is worth at best ~200B $ - or 1/10 of what it is now.

Scenario 2 - they maintain current revenue and the exceptional margins - the company would be worth ~1T - or 1/2 of what it is now.

Scenario 3 - they current growth rate (based on past 12 months) continue for ~5 years. This is the case the company is worth ~2T $.

But they are in a business where most money come from a handful of customers, all of which are working on similar chips - and given the sums in play now, the incentives are *very* strong.

My opinion, is that the company is already priced for perfection - basically the current price reflects the perfect scenario. I struggle to see any upside, unless we have AGI in the next 5 years and it decides it can only run on Nvidia chips.

All of this is akin to Tesla in the past years. They grew from a small startup to a medium car maker - the % growth rate was huge of course - an amazing achievement in itself. But people projected that the % growth rate would continue - and the stock was priced accordingly. Reality is catching up on Tesla, even if some projections are still absolutely crazy.

Buying long-term put options on Nvidia now is extremely expensive - the stock was so volatile that the price you pay for those options almost annihilate any gains you could expect, even if the stock losses 50% in 12 months.

You got me curious about those 5-7 trillions. Where these numbers come from ?

You've also had three or four bona fide bubbles in that span, starting around 2017. First was Bitcoin along with the stock market as a whole (with Nvidia being one of the leading stocks of that bull market advance).

Then you had Tesla go parabolic and lots of people become rich. Then you had the whole post-COVID speculative mania.

The result of this has been extreme credulity by the average person. Today's keynote is the perfect summation of this phenomenon. I saw multiple people who almost certainly couldn't explain in any level of detail how Nvidia GPUs are used for training and inference, but rather rely on the secondhand talking points like CUDA that they've learned by watching Jim Cramer, watching this keynote with excitement and anticipating how much it would pump their shares or call options.

Contrast this with Steve Jobs keynotes from 15 years ago when Apple's best days were well ahead of them. Most keynotes were questioned, in some cases even mocked. When Tesla stock broke out, many people couldn't make sense of it. Ditto for cryptocurrencies. But now, taking their cues from those cycles, the average person wants to ride the next bubble to riches and is trying to catch the wave and so now believes every story attached to a rising asset price.

CEO's aren't blind to this and are using every opportunity to create favorable storylines. The leadup to a keynote like this carries with it an enormous amount of pressure to deliver. Hence a company like Nvidia leaning into generative artwork and straight up made up storylines like robot development.

At the end of the day, I'm afraid that there likely isn't all that much substance and the evidence is beginning to pile up that the megacap tech stocks have run out of ideas which is why they are laying off people en masse and appealing to the AI hype cycle to carry their stocks higher.

Consider that Nvidia has gone up 8x- 800%!- in just over a year. The cycles are moving faster and faster. I remember just a year ago when lots of people said Nvidia at $250 was insane. Now here we are with the stock at more than three times that level and most people are calling it cheap. The stock market seems to have in certain areas like semis, completely disconnected from the fundamentals and taken flight. Yes, Nvidia earnings have grown. But understand that this is all part of a positive feedback loop where tech CEO's are pressured by their competitors and shareholders to show that they are investing in AI. Thus they all talk about it on their earnings calls and spend massively. All of their stocks rise in unison as you have a market that increasingly looks like its chasing momentum stock trends up. Nvidia's moves of late have almost nothing to do with any fundamental developments in the company. It has been routinely trading upwards of $45 billion a day. The Friday before last that number was over $100 billion. These are absolutely insane figures. Compare that to Microsoft, the largest company by market cap in the world, which trades on average around $8 billion per day.

I think this is generally how bull markets end and I think we may be actually forming the top of the great bull market for the megacaps that began around 2010 but really hit its stride starting in 2017.

This doesn't make much financial sense. Since every T-Bill is held by someone the amount of T-Bill outstanding is completely determined by government issuance. And since the amount of T-Bill outstanding will only grow over time, no "money" will ever flow out of it. Now if you narrow you inclusion of T-Bill holders to a specific group of people the amount this group holds could certainly go up and down over time. But then I wonder how you know this group sold stocks to buy T-Bills, and why this is more significant than the action of their counterparties: every share they sold was bought by someone else after all.

It doesn't matter for anyone who's not microsoft, aws or openai or similar.

Jensen's comment about being first was such a dig to Emerald Rapids.

Apple shows no such thing. Apple, sells pretty, reliable and safe. A car is a car, but apple is a sports car, or a saloon. Vertical integration is the way they chose to deliver that, and pretty and reliable are all normal people care about.

Nvidia is gonna have to think long and hard about the "whole stack". 20 years ago they might have been able to pull a next, but right now anything that isnt LINUX is a rounding error, and they dont want to turn into SUN (no one at nividia is smart enough to make them the next sun).

Nvidia architecture + Nvidia os is not something that I see them pulling off for the datacenter.