Effectively every single H100 in existence now will be e-waste in 5 years or less. Not exactly railroad infrastructure here, or even dark fiber.
That which survived, at least. A whole lot of rail infrastructure was not viable and soon became waste of its own. There was, at one time, ten rail lines around my parts, operated by six different railway companies. Only one of them remains fully intact to this day. One other line retained a short section that is still standing, which is now being used for car storage, but was mostly dismantled. The rest are completely gone.
When we look back in 100 years, the total amortization cost for the "winner" won't look so bad. The “picks and axes” (i.e. H100s) that soon wore down, but were needed to build the grander vision won't even be a second thought in hindsight.
H100s are effectively consumables used in the construction of the metaphorical rail. The actual rail lines had their own fare share of necessary tools that retained little to no residual value after use as well. This isn't anything unique.
How long did it take for 9 out of 10 of those rail lines to become nonviable? If they lasted (say) 50 years instead of 100, because that much rail capacity was (say) obsoleted by the advent of cars and trucks, that's still pretty good.
Records from the time are few and far between, but, from what I can tell, it looks like they likely weren't ever actually viable.
The records do show that the railways were profitable for a short while, but it seems only because the government paid for the infrastructure. If they had to incur the capital expenditure themselves, the math doesn't look like it would math.
Imagine where the LLM businesses would be if the government paid for all the R&D and training costs!
What killed them was the same thing that killed marine shipping — the government put the thumb on the scale for trucking and cars to drive postwar employment and growth of suburbs, accelerate housing development, and other purposes.
The age of postwar suburb growth would be more commonly attributed to WWII, but the records show these railroads were already losing money hand over fist by the WWI era. The final death knell, if there ever was one, was almost certainly the Great Depression.
But profitable and viable are not one and the same, especially given the immense subsidies at play. You can make anything profitable when someone else is covering the cost.
National infrastructure is always subsidized and is never profitable on it's own. UPS is the largest trucking company, but their balance sheet doesn't reflect the costs of enabling their business. The area I grew up in had tarred gravel roads exclusively until the early 1980s -- they have asphalt today because the Federal government subsidizes the expense. The regulatory and fiscal scale tipped to automotive and to a lesser extent aircraft. It's arguable whether that was good or bad, but it is.
State-level...? You're starting to sound like the other commenter. It's a big world out there.
> National infrastructure is always subsidized
Much of the network was only local, and mostly subsidized by municipal governments.
Actually, governments in the US rarely actually provided any capital to the railroads. (Some state governments did provide some of the initial capital for the earliest railroads). Most of federal largess to the railroads came in the form of land grants, but even the land grant system for the railroads was remarkably limited in scope. Only about 7-8% of the railroad mileage attracted land grants.
Did I, uh, miss a big news announcement today or something? Yesterday "around my parts" wasn't located in the US. It most definitely wasn't located in the US when said rail lines were built. Which you even picked up on when you recognized that the story about those lines couldn't have reasonably been about somewhere in the US. You ended on a pretty fun story so I guess there is that, but the segue into it wins the strangest thing ever posted to HN award. Congrats?
This remains to be seen. H100 is 3 years old now, and is still the workhorse of all the major AI shops. When there's something that is obviously better for training, these are still going to be used for inference.
If what you say is true, you could find a A100 for cheap/free right now. But check out the prices.
Edit: https://getdeploying.com/reference/cloud-gpu/nvidia-a100
Bulk pricing per KWH is about 8-9 cents industrial. We're over an order of magnitude off here.
At 20k per card all in price (MSRSP + datacenter costs) for the 80GB version, with a 4 year payoff schedule the card costs 57 cents per hour (20,000/24/365/4) assuming 100% utilization.
This is definitely not true, the A100 came out just over 5 years ago and still goes for low five figures used on eBay.
Are we? I was under the impression that the tracks degraded due to stresses like heat/rain/etc. and had to be replaced periodically.
I am an avid rail-to-trail cycler and more recently a student of the history of the rail industry. The result was my realization that the ultimate benefit to society and to me personally is the existence of these amazing outdoor recreation venues. Here in Western PA we have many hundreds of miles of rail-to-trail. My recent realization is that it would be totally impossible for our modern society to create these trails today. They were built with blood, sweat, tears and much dynamite - and not a single thought towards environmental impact studies. I estimate that only ten percent of the rail lines built around here are still used for rail. Another ten percent have become recreational trails. That percent continues to rise as more abandoned rail lines transition to recreational use. Here in Western PA we add a couple dozen miles every year.
After reading this very interesting discussion, I've come to believe that the AI arms race is mainly just transferring capital into the pockets of the tool vendors - just as was the case with the railroads. The NVidia chips will be amortized over 10 years and the models over perhaps 2 years. Neither has any lasting value. So the analogy to rail is things like dynamite and rolling stock. What in AI will maintain value? I think the data center physical plants, power plants and transmission networks will maintain their value longer. I think the exabytes of training data will maintain their value even longer.
What will become the equivalent of rail-to-trail? I doubt that any of the laborers or capitalists building rail lines had foreseen that their ultimate value to society would be that people like me could enjoy a bike ride. What are the now unforeseen long-term benefit to society of this AI investment boom?
Rail consolidated over 100 years into just a handful of firms in North America, and my understanding is that these firms are well-run and fairly profitable. I expect a much more rapid shakeout and consolidation to happen in AI. And I'm putting my money on the winners being Apple first and Google second.
Another analogy I just thought of - the question of will the AI models eventually run on big-iron or in ballpoint pens. It is similar to the dichotomy of large-scale vs miniaturized nuclear power sources in Asimov's Foundation series (a core and memorable theme of the book that I haven't seen in the TV series).