Hacker News Reader

Comment by barrkel | Hacker News Reader

What makes you think a website with "AI" is a big product?

IMO AI is positioned to be a commodity, and that's how Meta is approaching it, and of course doing their best to make it happen. I don't think, on the basis of what we've seen, that there is a sustainable competitive advantage - the gap between closed models and open is not big, and the big players are having to use distilled, less-capable models to make inference affordable, and faster.

I think it's probably clear to everyone that we haven't seen the killer apps yet - though AI code completion (++ language directed refactoring, simple codegen etc.) is fairly close. I do think we'll see apps and data sets built that could not have been cost-effectively built before, leveraging LLMs as a commodity API.

Realtime voice modality with interruptions could be the basis of some very powerful use cases, but again, I don't think there's a moat.

View on HN

What makes you think AI will become a commodity?

In 25 years, nobody has been able to compete with Google in the search space. Even though search is the best business model ever. Because search is so hard.

AI is even harder. It is search PLUS model research PLUS expensive training PLUS expensive inference.

I don't think a single company (like Meta) will be able to keep up with the leader in AI. Because the leader might throw tens of billions of dollars per year at it, and still be profitable. Afaik, Meta has spent less thatn $1B on LLAMA so far.

We might see some unexpected twist taking place, like distributed AI or something. But it is very unclear yet.

> What makes you think AI will become a commodity?

Because it already is. There have been no magnitude-level capability improvements in models in the past year (sorry to make you feel old, but GPT-4 was released 17 months ago), and no one would reasonably believe that there are magnitude-level improvements on the horizon.

Let's be very clear about something: LLMs are not harder than search. The opposite is true: LLMs, insomuch as it replaces Search, made competing in the Search space a thousand times easier. This is evidenced by the reality that there are at least four totally independent companies with comparable near-SOTA models (OpenAI, Anthropic, Google, Meta); some would also add Mistral, Apple Intelligence is likely SOTA in edge LLMs, xAI just finished a 100,000 GPU cluster, its a vibrant space. In comparison, even at the height of search competition there were, like, three search engines.

LLM performance is not an absolute static gradient; there is no "leader" per se when there are a hundred different variables upon which you can grade LLM performance. That's what the future looks like. There are already models that are better at coding than others (many say Claude is this), there will be models better at creative writing, there will be an entire second class of models competing for best-at-edge-compute, there will be ultra-efficient models useful in some contexts, open source models awesome at others, and the hyper-intelligent ones the best for yet others. There's no "leader" in this world; there are only players.

Yes, and while training is still expensive governments will start funding research at universities.

Search requires a huge and ongoing capital investment. Keeping an index online for fast retrieval isn't cheap. LLMs are not tools for search. They are not good at retrieving specific information. The desired outcome from training is not memorization, but generalization, which compresses facts together into pattern-generating programs. They do approximate retrieval which gets the gist of things but is often wrong in specifics. Getting reliable specifics requires augmentation to ground things in attributable facts.

They're also just not very pleasant to interact with. You have to type laboriously into a text box, composing sentences, reviewing replies - it's too much work for 90% of the population, when they're not trying to crank out an essay at the last moment for school. The activation energy, the friction, is too high. Voice modalities will be much more interesting.

Code assistance works well because code as text is already the medium of interaction, and even better, the text is structured and has grammar and types and scoped symbols to help guide generation and keep it grounded.

I suspect better applications will use the LLM (possibly prompted differently) to guide conversations in plausibly useful directions, rather than relying on direct input. But I'm not sure the best applications will have a visible text modality at all. They may instead be e.g. interacting with third party services on your behalf, figuring out how they work by reading their websites, so you don't have to - and it's not you doing the text interaction with the LLM, but the LLM doing text interaction with other machines.

>LLMs are not tools for search

I've used them for search. They can be quite good sometimes.

I was trying to recall the brand of filling my dentist used, which was SonicFill and ChatGPT got it straight away whereas for some reason it's near impossible to get from Google.

For sure, they are good for associative and analogy searches for well connected points in concept space, but leaf nodes are totally pulled out of the ether.

E.g. you can get great translation of source code from one language to another, but without extra effort, a chunk of API methods are going to be total fiction.

Or you can search for a good day trip to make when a tourist, and it'll get the major landmarks just fine, but e.g. restaurant recommendations are probably going to be made up.

Everybody seems to think AI in 10 years will be like AI now. But summarizing a PDFs and completing code is not the end of the line. It's just the beginning.

Let's look at an example of how we will use AI in the future:

    User: Where are my socks?
    AI: The red ones?
    User: Yes
    AI: You threw them away last week because they had holes.
    User: I see. On my way from work, where can I buy a pair of the same ones?
    AI: At Soandsoshop in Soandsostreet. It adds 5 min to your route.
    User: Great, let's go there later.
    AI: I can also just pick them up for you right now if you like.
    User: Nah, I would like to check some other stuff in that area anyhow.
    AI: Ok, I'll drive you there in the evening.

You still need search for that. Even more detailed search, with all items in all stores around the world. And you need an always on camera that sees everything the user does. And a way to process, store, backup all that. We will use way bigger datacenters than we use today.

Google Search wouldn't be reliable enough for that tho

Because AI is like software. Developing it is expensive, but the marginal cost of creating another copy is effectively zero. And then you can run it on relatively affordable consumer devices with plenty of GPU memory.

Search is also software. It did not move to consumer devices.

Search is more about data than software. And at that scale, the cost of creating another copy is nontrivial. LLMs are similar to video games in size, and the infrastructure to distribute blobs of that size to millions of consumer devices already exists.

Search is more about data, LLMs are somewhere between those two.

AI is a commodity right now, or at least - text. I just realized when paying the bills this month I got 1kg of cucumbers and a few KBs of text from opanai. They literally sell text by the kilo.

AI (of the type that OpenAI is doing) already is a commodity. right now.

So the question would be "what makes you think AI will stop being a commodity?".

Search needs to constantly update its catalog. I‘d say there are lots of AI use-cases that will (eventually?) be good for a long while after training. Like audio input/output, translations, …

Reply on news.ycombinator.com