Hacker News Reader

Comment by paulpan | Hacker News Reader

Not if Microsoft kills itself in the process? At best it's a Pyrrhic victory as both companies sink enormous infra costs to outdo one another. Worst case could be like Nokia acquisition debacle a decade ago when Microsoft desperately tried to re-break into mobile and failed.

There was a HN thread on this a while ago where consensus was serving cost ~$0.03-0.05 PER query, which is absolutely huge. Adds more credence to OpenAI's new subscription model of $20/month. https://news.ycombinator.com/item?id=34390123

View on HN

It's hard for Google now i think. It has really bad reputation on many things, which are hard to re-earn.

I myself no longer use Google search, and less Gmail (mostly for throwaway content).

Google gonna have a hard time to earn reputation in their services. They lost the "trust" war.

Bing also has a bad reputation.

Bing has never been a relevant search engine to earn a reputation, in my opinion. Microsoft has a bad reputation, but I'd say it changed a LOT in the past 10 years and has earned back a non-negligible chunk of trust.

Bing does have a reputation, people know what Bing is, and generally it is not a positive impression.

Microsoft can change that, but so can Google improve their reputation.

This is exactly John Bull's trust thermocline [0]

[0] https://nitter.lacontrevoie.fr/garius/status/158811531012453...

It's ~$0.03 per query if you were to buy the compute from AWS/Azure. If you own the datacenters it's probably more around $0.0003, perhaps less.

The profit margins of cloud providers are nowhere close to that. GCP and Azure currently lose money for Google and Microsoft, AWS is profitable but their profit margins are absolutely not 100x.

Microsoft, in this case, is pouring billions into this, and the industry at large, from Alphabet to LAION is in an arms race. They will optimize it beyond belief, x100 is rather low, come to think of it. I fully expect to be able to run your own ChatGPT on one DGX H100 unit (640GB GPU memory) in 18 months or less.

> GLM-130B, a model comparable with GPT-3, has 130 billion parameters in FP16 precision, a total of 260G of GPU memory is required to store model weights. The DGX-A100 server has 8 A100s and provides an amount of 320G of GPU memory (640G for 80G A100 version) so it suits GLM-130B well.

https://github.com/THUDM/GLM-130B/blob/main/docs/low-resourc...

There is surely lots of room for improvement [1].

[1] "Use GPT-3 incorrectly: reduce costs 40x and increase speed by 5x", https://news.ycombinator.com/item?id=34702349

> GCP and Azure currently lose money for Google and Microsoft

100% not true for Microsoft. Just listen to the most recent earnings call or read the analyst reports.

Eg https://www.nasdaq.com/articles/microsoft-nasdaq:msft:-cloud... “In fiscal Q2, Microsoft Cloud’s gross profit margin even expanded by about 2% year-over-year to 72%. When you have a business with such high margins growing at 20%+, profitability is set to expand considerably even if revenues in other segments fail to rebound”

> gross

Being the key word…

> average is probably single-digits cents per chat; trying to figure out more precisely and also how we can optimize it

- Sam Altman

https://twitter.com/sama/status/1599671496636780546

At that point, December 5th, 2022, they were still a somewhat regular Azure user. The $10 billion deal from Microsoft came a month later, January 10th, 2023.

No, they’ve worked together for years and been using Azure exclusively.

Source: https://blogs.microsoft.com/blog/2023/01/23/microsoftandopen...

Keyword being “extend” not “start”.

Well minimally Microsoft will bake it into their enterprise offerings, perhaps as a paid add-on or a throw-in to the higher tier subscriptions-- which are encroaching upon 100 dollars per user per month-- in order to move paying customers up the funnel.

Those kinds of analysis are total bs. Real cost is most likely a lot lower.

Reply on news.ycombinator.com