TSMC warns AI chip crunch will last another 18 months
theregister.com
theregister.com
I can't wait for copackaged optics, which will have a very similar manufacturing pipeline issue, to turn the NPU manufacturing pipelines into hell, that's surely coming and will stall performance networking for a solid year or two if CPO takes off.
There is no way in hell all these expensive fabs will pay for themselves if chips are too cheap. So no one (especially not TSMC) will risk oversupply by building too many fabs.
I keep hearing this - but I've seen no evidence of any successor board. Which board would you say is now the go-to board for hobbyists?
The price/performance (+size) is about the same as for the Pi, the power draw is not as bad as with older Intel/AMD machines, and your I/O options and x86 software compatibility is vastly better.
The only drawback is that you're buying used instead of new, and that your starting price for a small home lab (albeit one that starts at 2-3x the power of a single Pi) is larger, which harms students and other makers on a shoestring budget.
While this isn't exactly the case here since it is Hi-End packaging that is the problem. But I am still happy to read the statement on HN as I have been ranting about 99% of comments giving little to zero credit to current state of art chip manufacturing and cost.
CM4s are still being sold for hundreds of dollars on eBay.
Everyone and their dog is investing like crazy. Intel and TSMC are both setting up fabs in Silicon Saxony (although the question remains on how productive these will be, given the issues with the far-right in Saxony), TSMC is also expanding capacity in Arizona, and Samsung is building one in Texas.
There will be no oversupply, not for the next few years. Demand from automotive, hyperscalers, AI and pent-up consumer demand is ludicrous.
What does that have to do with anything?
California has plenty of neonazis itself. The western world in general has no lack of far right extremists.
[1] https://www.zeit.de/gesellschaft/zeitgeschehen/2018-10/recht...
This article [1] claims that there were 399 reported assaults associated with left-wing extremists, compared to 1013 associated with right-wing extremists.
Edit: and when it comes to property damage and arson, the far left is worse than the far right by about 3:1 [2]. This may be particularily relevant for companies who want to start new businesses.
[1] https://www.dw.com/en/left-wing-extremism-in-germany-how-muc...
[2] https://www.statista.com/statistics/963069/right-wing-left-w...
It seems like most of the US has this problem. And yet rural IT always seems to find a way. I’m in Lake Saint Louis with gigabit symmetric fiber, which is a dream. We were backwater for awhile, but post COVID a lot of talent can be sourced remotely, at least for software.
Is a fab really so regional? I could imagine them having a team in Taiwan who they work with remotely. But I know very little about fab plants, and it sounds like you have a better idea of their engineering and IT requirements.
I used to work at Groq, a hardware company, and we were fully remote. So I’ve seen this work in practice. (It’s a tossup whether Groq will succeed, but that’s a separate conversation.)
The US still benefits from latent good marketing/rep in the 80s/90s, and projection of military power in decades past. For example, until relatively recently (the Trump era and especially how it ended disillusioned many), South Koreans still looked to the US as their model of the developed world. If you'd send your children anywhere to get educated or marry, it would be there. Inner benchmarks were done against the US. The rose-tinted glasses were enough to convince many to move without checking too carefully what the place they'd be moving to would actually be like.
This is changing, and more recently more attention is being paid to Europe, partially because the national conversation has shifted more toward quality of life, birthrate problems, aging population, etc. and there's more interest in social systems, healthcare, family planning support and so on.
Still, Koreans have a lot of memes about this or that place in Europe being racist in this or that way (and of course, quite lot of it is true), so it remains a big impediment.
That seems like a 'slight' overstatement. Saxony is also full of crypto-communists (third highest polling state for Linke)...
> The main problem isn’t staffing the plant with human assembly and packaging
So why on earth are these companies building they are fabs there instead of some other place in Germany? I mean they can't be that stupid to not understand all of this?
[1] https://www.spiegel.de/wirtschaft/fachkraeftemangel-bloss-ni...
Do they have the same issues?
I assume the open border facilitates travel somewhat.
Also I don't think manufacturing cost is the biggest of issue, because the margins are significantly higher than consumer GPUs which are in oversupply now for Nvidia. It's that Nvidia has to commit years in advance for the capacity.
At minimum, LLMs work as a better human-software interface medium, which means they're going to be infused in almost everything non-technical consumer-facing.
Even absent any other breakthroughs, that seems like a huge consumer of AI hardware capacity.
In Feb they artifically constrained 4090 volume to try and push 4080s no one wanted. The 4070 was dead on arrival and had production cut by April.
They've near stopped production now - theoretically because the AI stuff is so lucrative, but if they were really "shipping a ton of units" of the 40 series and didn't have excess inventory i doubt they'd have done that.
- https://www.dexerto.com/tech/nvidia-rtx-4070-production-repo... - https://www.dexerto.com/tech/nvidia-rtx-40-series-production...
Rarely does Nvidia sell a GPU for 2000 dollars or more, and when they do its relatively low volume.
An H100 sells for tens of thousands of dollars and is sold out to 2024 - companies are literally fighting over them - I can absolutely see conditions might arise Nvidia might want to use some fab capacity for their much more profitable H100 parts vs consumer GPUs.
If they were to give up their consumer gpu lead and let AMD overtake them that will lead to more AMD support in ML, enabling AMD to eventually compete in the server gpu space too.
I think many, maybe even most, would claim this is now best done on cloud services, like huggingface.co.
Stable Diffusion blew up when people are able to run it at home.
The moat is their high quality libraries for speeding up common operations on Nvidia gpus.
AMD refuses to invest in ML and build the same libraries. Their stack is horribly buggy to the point of being unusable. It takes forever for their ML stack to get support for their new GPUs, even the crappy version they always produce.
If you start having physical space guarantees, then the calculus changes...
"The break-even point for a desktop vs a cloud instance at 15% utilization (you use the cloud instance 15% of time during the day), would be about 300 days ($2,311 vs $2,270):"
And that was written before the current GPU crunch and assumes availability in the cloud, which currently is not a given at all.
[1] https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni...
It's good they are selling shovels while there's a gold rush, but discarding the rest of your business to focus on a specific shovel that's currently popular may not end well.
I mean, we can just go ahead say "Intel" instead of beating around the bush.
They may end up in a position similar to US Steel or Standard Oil at the start of the 20th century if they continue to meet market demand as they do now, and AI is goes on to have the economic impact that steel and oil had in the last century.
——
Being a bit more fair, yes a bubble did burst then, but it was a “too much, too quick” type of bursting, not a “this whole thing is a scam” type.
(And as far as I can tell this current craze is entirely predicated on LLMs/transformers.)
llm companies raise lots of money on crazy evaluation, the question is if they will be able to build enough revenue stream to justify further money injections.
Atoms are more difficult than bits, so physical applications (e.g. in factories, robotics) will lag. I also expect specialist models like AlphaFold slowly finding their niches over time.
> significant, and one that opens up lots of new interesting applications as well as invites lots more R&D — but like the neural net it’s not gonna fundamentally transform society or industry.
Yeah, and semiconductor electronics that grad-students built from of germanium by painstakingly pressing them together are very interesting and worth additional R&D but won't transform society either.
LLMs are "language I/O interfaces", and as such there isn't a lot of value they can create alone (even most of the currently proposed uses are bullshit). But it's quite likely that they'll be there on a lot of advanced technology helping it do what you meant.
Still even though both will be there in the transformative technology, and even though the technology may not even be useful without them, they are not central pieces on most of it.
This whole fabrication thing is a biiig chokepoint.
This design is quite non-trivial even today. It took several of the brightest minds of all time to get it just right back then (like von Neumann).
Even with modern tools, you still need a team of quite capable engineers/scientists to pull something like that off, even if you do have the plutonium.
The little boy design is easier to engineer, but bomb grade uranium is even harder to produce than plutonium.
The machining bit so, uaing proper machinery, is easy. And the necessary engineers and technicians readily available.
What makes it easier, though, is that the general principle is relatively well known, so one could always brute force the solution using computer simulations, even if unable to do the math by hand.
The cost of the subscriptions to the design tools provided by Cadence, Synopsis, Siemens (Mentor) and a few others is ridiculously high and it can contribute with many millions of $ to the design cost of a state-of-the-art chip.
The design tools themselves have been improved extremely slowly and they are of a quite low quality for so expensive professional tools. In an open-source environment it would be easy to make much better alternatives.
The reason that only a few timid attempts exist to provide alternatives for only some parts of a complete toolchain, is that the semiconductor manufacturers, like TSMC, do not provide public information about the exact data that they need as input for manufacturing, about the design rules that must be implemented for valid designs and about the characteristics of their processes that must be known for simulations.
All this information is available only to big companies like Synopsis, Cadence etc. and this perpetuates their monopoly and their ability of extracting huge amounts of money with minimal software development work, from all the companies that design ICs.
While the companies that design ICs are not happy for this huge rent that they pay for the design tools, they are also extremely risk-averse and they do not want to make any change in their design workflows, no matter how small, that might increase the risks of failure in a designed chip. Because of this there are no chances of replacing the incumbents with open-source tools, at least not in the big established companies that design chips.
There are FOSS IC design tools but nothing that can match Synopsis, Mentor and Cadence and the semi industry isn't gonna start dealing with FOSS jank and destroy their productivity and competitiveness just to save a few milion in licensing costs to those 3 companies, which are peanuts in the end.
They're like the John Deer of semi tools, expensive to own and run, but necessary to be competitive and get the most out of this field.
Plus, the proprietary stuff is still the standard-cell libraries of TSMC, Samsung, Glo-Fo etc. They will never open source this.
>and this perpetuates their monopoly and their ability of extracting huge amounts of money with minimal software development work,
Oh, you mean just like Microsoft, Google, Apple, etc?
Because they sell ads and monetize their users instead, which is a pipe of infinite money.
Why does it matter to you how much they're charging? It's not communism. It's their honest labor and they're allowed to charge as much as they want for it, same how you're allowed to ask for as much money as you want for your salary negotiations.
So please excuse traditional engineering companies for trying to make a living by monetizing their hard earned IP like every single company out there, and not selling ads like big-tech.
The world still needs traditional engineering companies to exist to progress, we can't have a society where everyone only works by selling ads to one another.
If you feel like they're price gouging, then feel free to write your own EDA tools and sell them for cheap, I'm sure the market is open to paying much less and saving some money.
However most foundries, like too many other businesses today, are not interested in new customers, unless they already are very big companies, which is one of the factors that has resulted in the extraordinary reduction in the number of companies that compete in every field, in comparison with the situation from a couple of decades ago.
[1] https://en.m.wikipedia.org/wiki/Chip_War:_The_Fight_for_the_...
Open sourcing software works because some many people already have the hardware (a computer)
If you’re trying to draw parallels between this is and, say 3D printing, you will not find many similarities. PCB fabrication is also a a different beast but a lot of people think it’s as easy as finding a fab house that takes gerber files and creates a board for them.
If we want something that more closely resembles how we develop software, we probably will need to develop processes that enables us to flash FPGAs like MCUs, make them fully read only, and allow the customer to etch their own marking onto the chip. There facilities that produce ASICs like this already, but I can see something more standardized becoming more successful for general purpose. This does however require that we have a foundry create the original chips, the difference here is that they are producing the same standardized chip for everyone which would allow any foundry to make compatible alternatives.