SearchGPT Prototype | Hacker News Reader

SearchGPT Prototype

openai.com

134 pointsminimaxir28 comments

My condolences to OpenAI-dependent developers who just discovered their product was a feature.

https://github.com/ItzCrazyKns/Perplexica https://github.com/miurla/morphic

LLM frontends for search engines have been commoditized for a while now, this doesn't seem to add anything new (at least technologically)

Yeah, they seen how their product is used and now expanding the use case. Most of their userbase don’t know these tools, and looking for a better search than google.

Does it add much? I currently include "don't search web unless I ask" in my profile instructions. Then, when I want it to search the web, I just include "please search web" in my prompt. It works every time.

> We’re launching to a small group of users and publishers to get feedback. While this prototype is temporary, we plan to integrate the best of these features directly into ChatGPT in the future.

This is interesting. Right now there is a big distinction between how I use an LLM chat bot and a search engine. I'll have to get used to them being merged into one thing

Yeah it would suck if they decided to add search features into ChatGPT without a way to turn it off. There are lots of use cases for a language model on its own without all the extra noise. I really hope they keep them separate.

I presume that in terms of the OpenAI Assistants API (i.e. what you’d be using if you have a specific “use case”), search will be just another built-in tool (like the code interpreter) that an Assistant can be told to invoke.

(And “SearchGPT” is probably just such an Assistant, with the currently-hidden tool forced on.)

Announcement of an announcement (waitlist), gee this is 2017 crypto all over again

They haven't even shipped their last big thing yet, Sora was announced in February.

They seem to be slow-walking it with a select few users like with that Toys'R'Us ad. Probably to figure out how to sell it without risking regulation when it's inevitably used against politicians.

It's also still ambiguous how much compute it takes to run, and by extension how much it will cost to use.

They might be scrambling to optimize it to the point that it's feasible to ship.

Perplexity it was nice to know you

Phind too maybe. Though Phind can be used without log in, it starts giving you a fake “at capacity” modal and forces you to sign in after enough queries. Funny how they are no longer “at capacity” when I clear my cookies.

I would assume any product built on llm that has 10 million MAU, openai will go after. On the LLM side the gap is close between openai, anthropic and now meta. OpenAI might start to focus on products that’s built on its technology even if it’s UI/UX frontend.

ChatGPT is the new amazon, watching all of the transactions in their platform to decide what products they want to create next

Perplexity has some competition!

Lol more like, Perplexity has a terminal diagnosis.

Doing RAG using like.. prompt hacking and text embeddings + vector store when you have no access to the underlying model, nor ability to fine tune the generation for RAG, will fail. It will fail in an epic fashion compared to doing RAG the right way.

What do I mean by RAG the right way? The RAG term has been overloaded.

There's RAG that's just kind of bolted onto the LLM after it's been fine tuned for instruction following, and then there's RAG where the document/fact retrieval is a part of the LLM itself that is differentiated and optimized.

Almost everyone is doing the first "hacky" kind of RAG, but Meta published in 2020, the "correct" way to do RAG, where you include a neural retriever in the feedback loop.

Almost no one is doing this because it's more expensive (requires fine tuning the model), but will produce much better results than doing "bolted-on" RAG.

https://ai.meta.com/blog/retrieval-augmented-generation-stre...

Perplexity does not have access to any GPT model weights. It's unlikely they'll be able to compete on quality.

It's game over.

I'm not sure why the assumption of "will produce much better results". The fine tuning is not that predictable. Maybe some documents are remembered, maybe not. Maybe the document markers are preserved, maybe they fail. And adding anything new risks destroying existing data and is expensive.

Compare that to the vector + graph search, which is almost free to add to (if you're searching the internet, you're adding N documents per minute, not per days of training), repeatable, not affecting existing data. It would be cool to have a neural search, but how realistic is it without making it extremely fuzzy, forgetting and expensive?

Actually I disagree. I think the barrier to entry on a decent RAG system is very low. Embeddings have gotten so good, that retrieving chunks is going to be commoditized. Neural search was needed when embedding models were not good enough.

Didn't we have multiple open-weight frontier model announcements?

The reason they aren't dropping new models, is they are using their LLMs to embed in products, not as the end product. The APIs are a race to the bottom. They are going after Google.

They recently announced they are training their next big model, new models isn’t stopping it’s just taking more time to get more breakthroughs

The models are commoditized. There's no long term business model in a pure model company

Sure, hence projects like SearchGPT. OpenAIs main advantage is in mass distribution (Apple deal, etc)

Dupe (+ more discussion at): https://news.ycombinator.com/item?id=41071639

Now if Google copies it, or makes I'm Feeling Lucky default and you don't have millions for press coverage, you're done as a startup.

IN the mids of the entire AI bubble bursting, OpenAI decided to make some more news in the future tense, what a surpise

Google is so done

The already trash product could mabye compete with Perplexity, but it can't compete with the hype and user base openAI can conjure up

Reply on news.ycombinator.com