Big Tech groups say their $100B AI spending spree is just beginning
ft.com
ft.com
It means there's a mass over investment and over spend in companies trying to serve the demand at the time.
On it goes. These are just a few examples of where ML is already beginning to be applied and reaping rewards.
The same could happen again. It may be easier to find good uses for AI than to make large amounts of money with it.
Is it already 2000? Or is it in the beginning, 1995? 1998?
For some context, if you invested in Nasdaq in Jan of 1995 and did not sell until September 2001, you'd still be up by 86%. And if you invested at the absolute peak of the bubble, you'd still be up 250% in 2024.
Perhaps if you invested in Amazon or MS.
Meta, Anthropic, Google, Mixtral have demonstrated that OpenAI can be caught and matched.
Today, if you have a powerful enough GPU cluster, you can run an internal GPT4-level LLM by deploying Meta's LLama 3.1 405b.
If OpenAI's GPT5 is as big of a leap as GPT3 to GPT4, the hype will reach unprecedented level again in my opinion.
You also have to be willing to consume ungodly amounts of energy to run those GPUs. That seems like an important caveat while the conversation about climate change and unpredictable weather is still top of mind for so many people.
In 1999 people predicted that the Internet would change everything, in 2000 people called the Internet a flop and made fun of pets.com sock puppets, by 2010 the Internet had in fact changed everything.
All other things being equal they'd been better off investing their money or time into something that wasn't a bubble economy. This is basically broken window logic.
This feels different; there's actually some substance to the madness. Quite a few of the companies being funded are actually creating some pretty cool tech. And there's some real revenue potential as well; it's not just investment money keeping everything going. A good dot com era company reference would be companies like Google or Amazon that took the cash and got a lot of that tech making money for them even after the investment bubble burst. They also grabbed some of the smarter people at the same time. There are a few more examples. If you squint a little, you can see a few companies that are likely to be able to start raking in lots of cash soon that are at this point well funded.
Also a lot of the current investment money is being converted into GPU hardware. Which is of course nice for companies like NVidia, whom are probably a bit over valued currently. But the point is that hardware is tangible. Even if the companies that buy it go bust, the hardware just ends up in the hands of others. We're talking many millions of GPUs that are being deployed and that, like it or not, will be doing a lot of AI workloads for years to come for whomever ends up owning it. And there are a lot of smart people trying to make that hardware do all sorts of cool stuff. Hardware is a much better asset to have than useless websites. And I don't think a lot of the software is that bad either.
But today, deep-learning-flavored growth is a 10-year-old concept, and LLMs largely have leverage over existing (versus quite new) business models. There will probably be fewer new monopolies versus the dotcom era; in particular OpenAI has lots of competition.
There are two AI developments that I'm quite excited for:
1. Very large foundational models (such as GPT5-level LLMs)
2. Optimizations for smaller models, context size, inference speed, and multimodal LLMs
I think 2024 has been the year for #2, which is why the hype has died down a little. No splash big models that shock the world and make white collar workers shiver. But #2 is crucial to actually deploying LLMs to the masses.
What about these things excite you?
Just the tech hubris of growth at all costs or the hope of some use case that has yet to manifest?
> 1. Very large foundational models (such as GPT5-level LLMs)
Has work been announced on or a release date been set for a GPT5 or are you just excited it might happen? Why stop there? Go ahead, be excited about MilleniumGPT.
Is there evidence that bigger is better forever?
> 2. Optimizations for smaller models, context size, inference speed, and multimodal LLMs
To do what exactly?
Write fan letters to olympians for me?
Replace all of the real human friendships in my life?
To write an annoyingly verbose email from a bullet list that the recipient will be loathe to read and so will feed it into another agent to turn the email back into the same bullet list?
That they will annoy the hell out of you. ;)
1. No one would say the eye is conscious and yet its pattern matching is being re-used
2. The ML in the human brain re-uses the eye-cell wiring to pattern match....yes their was even a post either Friday or Sat here about that...
If we then re-tool the under-pining of AI, i.e. ML to match the new discovery we still get not AI but a new ML tool...
Or in short words find a fund index that is betting against the AI hype as the explosion will be massive.
Take the pessimistic view on LLMs for a moment. That this doesn't really work out and the expensive computers are a somewhat embarrassing misstep.
In that world, at least a few companies will find themselves with multibillion dollar high performance computers running on site. With capex and power numbers to scare the accountants and financial analysts. Not running LLMs.
That's a really fast computer. Computers can do stuff. If not LLMs, it's going to do other things.
To confidently sit out this capability acquisition round you need to be sure that LLMs are grossly overrated and also that all the competition will fail to find anything else to do with their GPU supercomputers. Oh, and that your own staff would also fail to find anything (else) useful to do with one.
I am completely comfortable expecting datacenter scale supercomputers with previously unimaginable compute to do interesting things. I wouldn't want to be the megacorp missing the revolution because I spent the money on dividends instead.
The things these can do is (a) memory bandwidth, Nvidia GPUs have been described as "the worlds most expensive memory controller". Generally very useful anywhere. (b) fast low precision floating point. Historically most of the uses of supercomputers wanted high precision, eg for physics etc but people are now experimenting with using lower precision given availability (c) fast networking. generally useful.
We were seeing a spread of GPU into other applications pre AI, like GPU driven databases, I think this stopped a bit just because of pricing and availability.
I've had to double check the low precision point qualifier. I expected MI300X to run f32 and f64 at the same speed (and it does) and nvidia to have slower f64 (which it does). It looks like H100 is 250TF at half, 60TF at float, 30TF at double. MI300X is 650TF at half, 80TF at float or double. Very fast F16 indeed.
(sourced from https://www.techpowerup.com/gpu-specs/radeon-instinct-mi300x... and https://www.techpowerup.com/gpu-specs/h100-sxm5-96-gb.c3974, I might have the wrong SKUs for comparison)
If so there could be another round of investment at a better performance per watt, and specialised hardware will be the key. They’ll all be building custom chips.
In that world, a tech company could arbitrarily invest in multibillion dollar high performance computers for no particular reason. Following the argument, with all that compute on site with nothing to do engineers would would find something interesting and all other companies that didn't invest would be missing out on that revolution.
So by the argument, any tech company could gain a competitive advantage by any non-strategic investment on the assumption that it will always work out. But of course, this only works if you have infinite money and infinite opportunity to speculate. As soon as your resources are constrained, then strategic choice becomes the dominant factor and your CEO has a lot of explaining to do.
If it's a mistake, your excuse is that your competition did it too and you wanted to guard against being left behind. If it works, everyone is happy.
If you're the only company making that speculative investment, it's great if it works and you might be fired by the board if it doesn't.
But I will take your point that you have more cover for making a bad decision if you know all your competitors are making the same bad decision. But you still missed the opportunity to not make a bad decision and therefore get ahead. The actual risk remains the same.
From the perspective of the people running a successful company, it's much more important to make easily defensible reasonable decisions than to make ambitious ones. You protect the capital in preference to maximising returns. Major gains are somewhat rewarded and major failures severely punished, with a comfortable baseline if you maintain the current positioning.
If you're not yet a successful megacorp, all the dials are turned in favour of risk because you need the reward. Lots of incomers doing riskier things seeking to overthrow the incumbent is roughly how we get a turnover of companies and a degree of overall progress.
I think this round is interesting because the incumbents have seen substantial competitive risk which could otherthrow them on an alarmingly short timescale (i.e. while the current leadership are still there), and that has induced otherwise fairly unlikely massive capex spend.
After hiring a lot of people, having systems built, and training data developed, they could replace large amounts of staff that used to need to do manual work those AI systems. Less manual moderation, more automated responses. It became "fashionable" to cut staffing levels, so companies followed suit.
Most tech leaders avoid talking about how much labor AI will replace. We don't know yet, but it likely follow trends of making junior roles harder to find while making seniors more valuable with a slight reduction in the overall workforce. This has been present in all industries
I think it is:
1. Over hiring from covid
2. A switch from developing traditional software to one that is LLM based
The interest rates mean that the now gets a higher priority, and software ready in 2030 or later a lower priority.
It is the same everyone does. It is not like these people will keep their plumber of home builder employed out of good will after the job is done. Or nanny around after their kids have fully grown...