Hacker News Reader

Comment by paulpan | Hacker News Reader

Most interesting tidbits: 1 - Sam Altman onstage all but confirmed the underlying LLM model as GPT-4 as he kept saying "next gen" without providing a name 2 - Microsoft is all-in by making this front-and-center of Bing, as well as deep integration into Edge browser

I'm most curious about how Microsoft is planning to pay for the infrastructure cost of serving the traffic.

View on HN

> all but confirmed the underlying LLM model as GPT-4 as he kept saying "next gen" without providing a name 2

It is not. It's still 3.5:

"Microsoft says these features are all powered by an upgraded version of GPT 3.5, the AI OpenAI language model that powers ChatGPT. Microsoft calls this the “Prometheus Model,” and says it’s more powerful than GPT 3.5, and better able to answer search queries with up-to-date information and annotated answers." (https://www.theverge.com/2023/2/7/23587454/microsoft-bing-ed...)

An upgraded version could be 4.0

LTE was technically called 3.9G

Its goal is to kill Google + Chrome i think. Profit comes later ?

Not if Microsoft kills itself in the process? At best it's a Pyrrhic victory as both companies sink enormous infra costs to outdo one another. Worst case could be like Nokia acquisition debacle a decade ago when Microsoft desperately tried to re-break into mobile and failed.

There was a HN thread on this a while ago where consensus was serving cost ~$0.03-0.05 PER query, which is absolutely huge. Adds more credence to OpenAI's new subscription model of $20/month. https://news.ycombinator.com/item?id=34390123

It's hard for Google now i think. It has really bad reputation on many things, which are hard to re-earn.

I myself no longer use Google search, and less Gmail (mostly for throwaway content).

Google gonna have a hard time to earn reputation in their services. They lost the "trust" war.

Bing also has a bad reputation.

Bing has never been a relevant search engine to earn a reputation, in my opinion. Microsoft has a bad reputation, but I'd say it changed a LOT in the past 10 years and has earned back a non-negligible chunk of trust.

Bing does have a reputation, people know what Bing is, and generally it is not a positive impression.

Microsoft can change that, but so can Google improve their reputation.

This is exactly John Bull's trust thermocline [0]

[0] https://nitter.lacontrevoie.fr/garius/status/158811531012453...

It's ~$0.03 per query if you were to buy the compute from AWS/Azure. If you own the datacenters it's probably more around $0.0003, perhaps less.

The profit margins of cloud providers are nowhere close to that. GCP and Azure currently lose money for Google and Microsoft, AWS is profitable but their profit margins are absolutely not 100x.

> GCP and Azure currently lose money for Google and Microsoft

100% not true for Microsoft. Just listen to the most recent earnings call or read the analyst reports.

Eg https://www.nasdaq.com/articles/microsoft-nasdaq:msft:-cloud... “In fiscal Q2, Microsoft Cloud’s gross profit margin even expanded by about 2% year-over-year to 72%. When you have a business with such high margins growing at 20%+, profitability is set to expand considerably even if revenues in other segments fail to rebound”

> gross

Being the key word…

Microsoft, in this case, is pouring billions into this, and the industry at large, from Alphabet to LAION is in an arms race. They will optimize it beyond belief, x100 is rather low, come to think of it. I fully expect to be able to run your own ChatGPT on one DGX H100 unit (640GB GPU memory) in 18 months or less.

> GLM-130B, a model comparable with GPT-3, has 130 billion parameters in FP16 precision, a total of 260G of GPU memory is required to store model weights. The DGX-A100 server has 8 A100s and provides an amount of 320G of GPU memory (640G for 80G A100 version) so it suits GLM-130B well.

https://github.com/THUDM/GLM-130B/blob/main/docs/low-resourc...

There is surely lots of room for improvement [1].

[1] "Use GPT-3 incorrectly: reduce costs 40x and increase speed by 5x", https://news.ycombinator.com/item?id=34702349

> average is probably single-digits cents per chat; trying to figure out more precisely and also how we can optimize it

- Sam Altman

https://twitter.com/sama/status/1599671496636780546

At that point, December 5th, 2022, they were still a somewhat regular Azure user. The $10 billion deal from Microsoft came a month later, January 10th, 2023.

No, they’ve worked together for years and been using Azure exclusively.

Source: https://blogs.microsoft.com/blog/2023/01/23/microsoftandopen...

Keyword being “extend” not “start”.

Well minimally Microsoft will bake it into their enterprise offerings, perhaps as a paid add-on or a throw-in to the higher tier subscriptions-- which are encroaching upon 100 dollars per user per month-- in order to move paying customers up the funnel.

Those kinds of analysis are total bs. Real cost is most likely a lot lower.

Exactly. That seems like the plan. They have huge cash to burn and for once, they seem to have an edge on Google in what has traditionally been google’s core business. Profits will follow along if they can divert the huge percentage of search engine and browser user-base to Bing and Edge.

If it was GPT-4 he would have said it. There were, however, references to an updated GPT 3.5.

With cash on hand and gained market share parlayed into ads and more cash likely

> Sam Altman onstage

Do you mean his interview 3 weeks ago?

https://www.youtube.com/watch?v=eVcJJZz9P4U

I guess GPT-4 needs its own launch event. :)

Reply on news.ycombinator.com