Hacker News Reader

Comment by throwaway55905 | Hacker News Reader

We've got to stop calling this a "Google Memo." That's a false narrative. It's just a random doc written by one of 140000+ employees.

> Google has been contacted for comment but it is understood that the document is not an official company memo. [1]

There are moats to products, but less so to pure language models trained on the same web-scale scraped data that many share.

Not all data is readily available to language models, and integration can be difficult.

A company that specializes in say AI for trash sorting likely still has a moat.

Microsoft integrating AI into Windows still has a moat (for Windows).

GPT-4 is ~200 Elo better than the next best semi-public Vicuna-13B in Chatbot Arena [2]. That is a non-zero moat - perhaps due to hosting larger models, training data, licensing, output postprocessing, etc.

[1] https://www.theguardian.com/technology/2023/may/05/google-en...

[2] https://lmsys.org/blog/2023-05-25-leaderboard/

View on HN

> GPT-4 is ~200 Elo better than the next best semi-public Vicuna-13B in Chatbot Arena [2]. That is a non-zero moat

Its a non-zero advantage.

A moat is something that inhibits someone from closing an advantage.

(Also, its odd that the biggest models, outside of the big vendor centralized ones, they are testing are 13B-14B when 30B-ish and 65B-ish versions exist.)

Perhaps it's not a moat.

However, if the advantage is due to things like inference infrastructure to support a massive model, that isn't easy to duplicate.

I would also say that the quality of these smaller models are good, but we also may not be measuring them correctly. Recent papers suggest that these smaller LMs dont fully capture ChatGPT quality in ways that may not have appeared with crowd worker ratings [1]. It's easy to have your inputs be inside a happy distribution for a paper but fail in the real world in ways that GPT-4 doesnt.

Lmsys would love to compare with bigger models but have limited resources. Contributions are welcome [2]

[1] https://arxiv.org/abs/2305.15717

[2] https://lmsys.org/blog/2023-05-25-leaderboard/#next-steps

OpenAI doesn’t make their own hardware, even in the fabless sense of “make.” So in what way is inference infrastructure a moat?

The moat (or the water/crocodiles in it) is the content that the company gathers in relation to the offering that is being defended. Microsoft has Github, which is a source of code that the model can operate upon, as well as the interactions/queries with the users. OpenAI is playing around with sharing content because of this. They want to build a moat and will use us to do it.

If someone just has an approach to solving the problem, i.e. code that does this that and the other, then there is no moat.

> We've got to stop calling this a "Google Memo." That's a false narrative. It's just a random doc written by one of 140000+ employees.

So it's a.. memorandum?

yes but "{person name} memo" is not the same as "Google Memo" (published and endorsed by the company)

> (published and endorsed by the company)

this is not implied by the name "Google memo".

It is a memo written by a Googler on internal systems that makes it a Google memo. Companies almost never officially sponsor internal memos escaping without PR and legal having a crack at the content.

What I'm really curious about is why you think this isn't a Google Memo, and why you think that's a false narrative.

There is a huge difference between a leadership-endorsed strategy memo like the Nokia/Elop "Burning Platform" memo (1), and a memo by a random engineer like Steve Yegge's Platform rant (2).

Random engineer memos can certainly still be influential, but they do not dictate company direction.

(1) https://www.engadget.com/2011-02-08-nokia-ceo-stephen-elop-r...

(2) https://gist.github.com/chitchcock/1281611

Anywhere I've worked, junior engineers have put together ridiculous memorandums, especially when they're on the way out. That does not mean that the opinions of the author align with the company's.

To me, "memo" implies it being for the business. If this was written by leadership or intended to be some sort of instruction to others, then I think it would be more reasonable to call it that.

But it wasn't. It was an opinion written down by an individual, for no other reason than to share their personal opinion. It was closer to an HN comment, not a document necessary as part of business operations.

> A company that specializes in say AI for trash sorting likely still has a moat

Is the most the trash data

(250,000 employees. Just about half of staff is TVC)

for others, TVC apparently means "Temporary, Vendor, or Contractor", aka non-FTE

for others, FTE apparently means "Full-Time Employee", aka non-TVC

was just gonna ask, thanks

Not random. A 'researcher at Google'.

Reply on news.ycombinator.com