Hacker News Reader

Comment by digitcatphd | Hacker News Reader

I agree with you at this time, but there are a couple things I think will change this:

1. Agentic search can allow the model to identify what context is needed and retrieve the needed information (internally or externally through APIs or search)

2. I received an offer from OpenAI to give me free credits if I shared my API data with it, in other words, it is paying for industry specific data as they are probably fine tuning niche models.

There could be some exceptions to UI/UX going down specific verticals but eventually these fine tuning sector specific instances value will erode over time but this will likely occupy a niche since enterprise wants maximum configuration and more out of box solutions are oriented around SMEs.

View on HN

It comes down to moats. Does OpenAI have a moat? It's leading the pack, but the competitors always seem to be catching up to it. We don't see network effects with it yet like with social networks, unless OpenAI introduces household robots for everyone or something, builds a leading marketshare in that segment, and the rich data from these household bots is enough training data that one can't replicate with a smaller robot fleet.

And AI is too fundamental of a technology that a "loss leader biggest wallet wins" strategy, used by the likes of Uber, will work.

API access can be restricted. Big part of why Twitter got authwalled was so that AI models can't train from it. Stack overflow added a no AI models clause to their free data dump releases (supposed to be CC licensed), they want to be paid if you use their data for AI models.

I don't think OpenAI have a moat in the traditional sense. Other players offer the exact same API so OpenAI can only win with permanent technical leadership. They may indeed be able to attain that but this is no Coca-Cola.

I wasn't referring to OAI, but rather:

1. Existing legacy players with massive data lock-ins like ERP providers and Google/Microsoft.

2. Massive consolidation within AI platforms rather than massive fragmentation if these legacy players do get disrupted or opportunities that do pop up.

In other words - the usual suspects will continue to win because they have the data and lock in. Any marginal value in having a specialized model, agent workflow, or special training data, ect. will not be significant enough to switch to a niche app.

It is indeed unfortunate and niches will definitely exist. What I am referring to is primarily in enterprise.

    > Agentic search

All you've proposed is moving the context problem somewhere else. You still need to build the search index. It's still a problem of building and providing context.

what do you think about these guys: https://exa.ai/

Crawling web data is ETL. I think the case stands: the winners in AI/LLM SaaS startup space are the ones that really do ETL well. Whether that's ETL is across an enterprise data set or a codebase.

The AI and LLM are just the the "bake" button. If you want anything good, you still have to prep and put good ingredients in.

I disagree, these search indexes already exist, they just need to be navigated much how Cursor uses agentic search to navigate your codebase or you call Perplexity to get documentation. If the knowledge exists outside of your mind it can be searched agentically.

To your first point, the LLM still can’t know what it doesn’t know.

Just like you can’t google for a movie if you don’t know the genre, any scenes, or any actors in it, and AI can’t build its own context if it didn’t have good enough context already.

IMO that’s the point most agent frameworks miss. Piling on more LLM calls doesn’t fix the fundamental limitations.

TL;DR an LLM can’t magically make good context for itself.

I think you’re spot on with your second point. The big differentiators for big AI models will be data that’s not easy to google for and/or proprietary data.

Lucky they got all their data before people started caring.

> Just like you can’t google for a movie if you don’t know the genre, any scenes, or any actors in it,

ChatGPT was able to answer "What was the video game with cards where you play against a bear guy, a magic guy and a set of robots?" (it's Inscryption). This is one area where LLMs work.

You described all of those things to some extent, as much as they apply to video games. No magic here.

“Playing cards against a bear guy” is a pretty iconic part of that game… that you, as a human, had the wherewithal to put into that context. Agents don’t have that wherewithal. They’d never come up with “playing cards against a beat guy” if you asked it “what game am I thinking of”

Let’s do another experiment. Do the same for the game I’m thinking of right now.

There were characters in it and one of them had a blue shirt, but that’s all I can remember.

LLMs are really good at 20 questions, so if you give it a chance to ask some follow-up (which they will do if given such a prompt) it will probably figure it out pretty quick.

Sure, so if I, as a human, play 20 questions and add all that context to an LLM, it can perform.

That’s true. It’s why these things aren’t useless.

I’m saying that LLMs arent able to make context for themselves. It’s why these agentic startups are doomed to morph into a more sensible product like search, document QA, or automated browser testing.

Reply on news.ycombinator.com