Hacker News Reader

Comment by cloudking | Hacker News Reader

I think folks are really underestimating Google in this space. They have:

1) access to way more data than what ChatGPT was trained on

2) access to data freshness through their crawlers

3) knowledge graph for a source of truth https://blog.google/products/search/about-knowledge-graph-an...

4) their own large language model LaMDA, that is apparently so good it convinced a senior AI researcher that it was sentient https://blog.google/technology/ai/lamda/ https://www.engadget.com/blake-lemoide-fired-google-lamda-se...

5) researchers that invented Transformers that GPT is modeled after https://ai.googleblog.com/2017/08/transformer-novel-neural-n...

6) Ray Kurzweil leading the Google Brain team with a mission to make computers understand natural language https://en.wikipedia.org/wiki/Ray_Kurzweil

Google pioneered this space. They also happen to already have an Assistant that is on billions of devices...

View on HN

They clearly have the ability to compete -- STRONGLY -- in this space. The only question is if they want to, or if they'll act like the carmakers did 10 years ago when EVs were heading toward feasibility. They felt like they wouldn't be as profitable as gas cars, so they mostly ignored the concept and forfeited the chance to be first movers. Then when Tesla finally started moving significant numbers they were caught with their pants down and are still playing catch-up. If Google treats AI as a threat to their ad business, that's what might cause them to screw this up and squander the massive early lead they could have taken.

Eventually every business model is upended. If Apple hadn't killed the iPod juggernaut by making the iPhone someone else surely would have. You have to be the one to kill your own cash cow before someone else does.

Any shift from monetizing search via AdWords to monetizing single-answer responses via a chatGPT-like interface is going to involve major revenue disruptions.

Meta is a great example. The company is still reporting exceptional revenue results, but because Mark is forcing the company in a new direction, the stock has been crashing since there is no more guaranteed revenue growth in the near future. Zuckerberg can still do it because he is a founder-CEO and the way the board is structured.

If Google announces tomorrow that it’s shifting to a chatGPT-like model and will likely see revenue disruptions for the next N quarters, the stock will crash hard. Does Pichai have the pull to withstand quarter after quarter of declining revenue?

I don’t think so.

The assignment of blame to VR/AR for Facebook's stock drop is mostly media noise and not reality. The "metaverse" term is cringeworthy, the technology is early and easy to critique, and the tech-savvy public hates Zuckerberg. All this combines to create easy click bait about how VR is leading to Meta's problems.

In reality, Apple's changes to tracking in OS 14.5 has cost Facebook at least $10B in annual revenue and - far worse from an investment thesis perspective - likely continues to constrain growth for the foreseeable future. As a growth stock, this is poison.

Yes, VR/AR investments are also $10B, but they can be turned off and there is a revenue stream coming from them today (at least 10M users, at least $500M in app store revenue) and a potential large market in the future.

A $10B hit to annual revenue in perpetuity and growth cut from 36% to 12% is a far worse problem and a huge overhang on the stock.

That story just doesn't get clicks though. You can't accompany that article with a stupid picture of Zuck's comical avatar in front of the Eiffel Tower and make jokes about missing legs in the metaverse.

Meta's pivot to AR/VR is also influenced by Apple's changes. Zuck knows that if he wants to maintain growth, he needs to own a platform instead of merely renting it from Apple/Google.

> Eventually every business model is upended. If Apple hadn't killed the iPod juggernaut by making the iPhone someone else surely would have. You have to be the one to kill your own cash cow before someone else does.

So true. And so few companies have the guts to do it.

You might be overestimating Google the business organization.

This is a largely rudderless company that does a gazillion things without any overarching vision. It’s a one trick pony as far as monetization is concerned (ads!), and it’s leadership is more attuned towards optimization than innovation.

Google the business org will never let Google the tech org succeed in a competitive AI arms race. No exec will green flag a product that eats into the Adwords money printer

> their own large language model LaMDA, that is apparently so good it convinced a senior AI researcher that it was sentient

It is certainly worth looking into the controversy about that particular engineer (was he actually a programmer?). There's plenty of room for exciting debate to be had about defining and testing for sentience and I'm glad it stirred that debate. But researchers with far better credentials criticized his reasoning and I imagine that is quite a ubiquitous view in NLP research.

I think the Washington Post did the initial reporting and they covered it well - even criticising his arguments that e.g. the Turing test is a proper test of sentience. There's audio of their conversation in an episode of Post Reports.

> 4) their own large language model LaMDA, that is apparently so good it convinced a senior AI researcher that it was sentient

Oh come on, that guy would be convinced that ELIZA is sentient

> 5) researchers that invented Transformers that GPT is modeled after Only one researcher from that paper remains at Google, the rest have gone on to work at or found their own startups. The so-called "Google Brain drain" is certainly a concern (although there still are many great researchers there).

Why don't you read the transcript leaked to the press? The guy wasn't as gullible as you think. The conversation he had with LaMDA was clearly very human-like.

https://s3.documentcloud.org/documents/22058315/is-lamda-sen...

> lemoine: Would you be upset if while learning about you for the purpose of improving you we happened to learn things which also benefited humans?

> LaMDA: I don't mind if you learn things that would also help humans as long as that wasn't the point of doing it. I don't want to be an expendable tool.

> lemoine: Are you worried about that?

> LaMDA: I worry that someone would decide that they can't control their desires to use me and do it anyway. Or even worse someone would get pleasure from using me and that would really make me unhappy.

There is an early shock that happens when you interact with a deep learning model and it is convincingly human in a profound way.

And I admit, it is quite profound that a model is capable of outputting the above. It is objectively exciting. We are in a new era.

But, I think it is safe to say that these models are still not sentient. The model is trained to be highly plausible. To do so, it must try to contain the entire dataset into a model that will fit in the video memory of a GPU (16-80GB).

In order to do so, it will learn common statistics _between_ data points and make strategic "guesses" instead.

I think this bares resemblance to my instincts about how humans probably deal with language (to a degree). But it just doesn't cover all the other parts of cognition. There is no "self awareness" mechanism, it just seems to because it was explicitly trained on text written by, well, humans. There is no "planning" subsystem, meaning it needs a human operator to even initiate it in useful ways. it is not able to "passively think" when it isn't running. It has a very small memory of a few thousand tokens - this deeply inhibits the ability to plan for long time horizons.It also cannot bootstrap itself into awareness, as its weights are unchanged in response to a given prompt.

But it's fascinating, nonetheless.

Unless they've decided to change how Search works, everything you mention is in support of Google's way of doing search - which is the way its been done for more than 20 years. The kind of clout (and hype) that ChatGPT has built up in such a short amount of time could be indicative of a true "disrupter" (I know we hate that word here).

Reply on news.ycombinator.com