Hacker News Reader

Comment by esafak | Hacker News Reader

I'd be surprised because (free) open source are continually closing the gap, exerting downward pressure on the price.

Meaning the real value is in infra. Running local is super expensive as you need to provision 100% and there are high table stakes. So people will tend to use a cloud offering. If the model doesnt differentiate then its how you mix the models, the capacity and uptime, the "GPT wrappers" etc.

I don't think it will be much of an issue for large providers, anymore than open source software has ever been a concern for Microsoft. The AI market is the entirety of the population, not just the small sliver who knows what "VRAM" means and is willing to spend thousands on hardware.

You can get open source models hosted for cheap too; e.g., through OpenRouter, AWS Bedrock, etc. You do not have to run it yourself.

> anymore than open source software has ever been a concern for Microsoft.

So a big concern then? (Although not a death sentence)

The modern Microsoft with Azure, Office360, etc is not much threatened by open source software. Especially with Azure, open source is a fantastic compliment which they would like the world to produce as much of as possible. The same with AI models. They would look at charge for AI hosting and services, at premium due to already being integrated in businesses. They are going to bundle it with all their existing moneymakers, and then just jack up the price. No sale needed, just a bump in the invoices that are flowing anyway.

They're definitely very threatened by open source - a lot of software infrastructure these days is built off of open source software. In the 2000s, it wasn't. It was Microsoft, MSS, COM, Windows server, etc all the way down. Microsoft has basically been earned alive by open source software, it's just hard to tell because they were so huge that, even taken down a few pegs, they're still big.

Even today, Azure and AWS are not really cheaper or better - for most situations, they're more expensive, and less flexible than what can be done with OS infrastructure. For companies who are successful making software, Azure is more of a kneecap and a regret. Many switch away from cloud, despite that process being deliberately painful - a shocking mirror of how it was switching away from Microsoft infrastructure of the past.

The phrasing was "has ever been" but even in the modern era, you're only looking at their winners.

Shouldn't something like Kubernetes or Android's flavor of open source be on the radar? Seems like there are plenty of large players that might turn their 4th place closed source API into a first place open ecosystem.

Hopefully we see enough efficiency gains over time that this is true. The models I can run on my (expensive) local hardware are pretty terrible compared to the free models provided by Big LLM. I would hate to be chained to hardware I can't afford forever.

The breakthrough of diffusion for tolken generation bumped down compute alot. But there are no local open sources versions yet.

Distillation for specialisation can also raise the capacity of the local models if we need it for specific things.

So its chugging along nicely.

They're not really free, someone still has to pay for the compute cost.

Reply on news.ycombinator.com