Hacker News Reader

Comment by est | Hacker News Reader

LLM is not only a service, it shook the fundation of Google's vision.

https://www.google.com/search/howsearchworks/our-approach/

> Google’s mission is to organize the world's information

yeah but you only organized the info, but GPT-3/ChatGPT was the first of is kind to actually "understand" the information.

View on HN

I'd say that LLMs are a different way of organizing the world's information through a kind of lossy compression into enormous probability distributions that can approximate the original information with enough fidelity to be useful for a lot of low-stakes applications.

That was a lot of information compressed into a single sentence, but I think you hit the nail on its head. Most things I read about ChatGPT makes me think that most people don't understand what an LLM really is though and make assumptions on what it can do and what it 'understands'.

That's because people look at what it can do rather than what it really is. Maybe what it really is doesn't really matter all that much except to 'AI purists'. But to the rest of the world if a company markets something as 'AI' they will treat it as such, including calibration of their response to the effects of that technology independent of how it works under the hood. Whether your job gets obsoleted by LLM or AGI doesn't really matter to people who couldn't tell the difference between the two and it shouldn't matter to them. Their voice deserves to be heard in this whole debate but the tech world has decided they don't really matter.

It’s worse: they look at what they anthropomorphically think it’s doing. Like looking at an animal picture classifier work and thinking it can see animals, or a watching a self-driving car and thinking it has some kind of simulation of a human driver in it.

I think the “AI purists” need to make a lot more noise about the fact that nobody knows what LLMs are doing.

There's a term for this anthropomorphic error in the neurosciences/psychology - a homonculus. Like your theory implies there's a little man in there doing the work. (homonculus also has other meanings in neuro, including cortical mappings, just sayin)

When I was doing my master's thesis, my industrial supervisor opined that a neural network "is really just a glorified lookup table". I think, given the succinctness of the characterisation, it's not too far off the mark.

Yeah this is a great description.

People are saying it's a "lossy jpeg" in a way that is disparaging and minimises how awesome LLMs actually are! They are a form of text compression that leverages the semantics of the text! Amazing!

And can approximate combinations of that information (hopefully with enough fidelity) that weren't present in the original training body. This is where the value lies, if all it did was spit back out what went in it really would be just a compression algorithm.

I'm not sure if that's the case or not.

In 2017 I could ask Google which episode of the Futurama was the one where the save the planet from a trash ball, and it could answer the question in natural language.

I suspect that Google X has had these, and other capabilities for a while even if they've never been shipped in any specific product. Or how about Duplex, where Google could call a restaurant to make a reservation, including negotiating times? That was also like 2017-2018.

These are only the public, polished, productized ideas. Google has bard, a response to chat gpt, probably only because they didn't realize that people actually wanted a chatbot. I think it was a product failure. How quickly they responded with something similar I think shows you this tech exists inside Google already.

Remember, OpenAI was created in part to help make AI accessible to people outside of Google/Microsoft. Who knows what's going on behind closed doors.

I don't know that it ruined it. However it creates a challenge for search which makes previous iterations of SEO spam look like child's play. So the signal to noise ratio is going to get massively worse. And if they don't give strong signal to users and continue with an experience increasingly suffocated by ads, users will go wherever the AI prompt is embedded with the least friction in their UI with the minimum acceptable results.

Which makes Microsoft relevant again because they own the lower level UI experience on a staggering percentage of screens.

I'd expect nothing but aggressively shoving clippy 2.0 into every surface Microsoft owns. And I expect they will be very light with the ads because they have the additional revenue streams to play a longer game on this if it hurts Google's market share.

Google doesn't really have the same luxury.

You left out the other 2/3 of the mission statement: "...and make it universally accessible and useful."

yeah the second half gets interesting. Making it accessible is not enough, LLMs can intepret it in different ways.

Besides for the fact that current LLMs don't understand anything, being first isn't part of that statement. Google also wasn't first to have a search engine, or a video sharing site, or an ad exchange, or an email service.

Reply on news.ycombinator.com