ChatGPT Opened a New Era in Search. Microsoft Could Ruin It | Hacker News Reader

ChatGPT Opened a New Era in Search. Microsoft Could Ruin It

wired.com

35 pointssharemywin23 comments

My personal prediction is that llm powered search will be as terrible as google search once people figure out how to game it. In a way probably worse because it will only give one answer instead of a list of answers that you can pick through.

Exactly. An LLM-based search engine has already amounted into being a more untrustworthy search engine, even worse than Google, with hallucinated results and results that are just plain sophistry from the AI with a lack of reasoning and explanation.

The only 'killer' use-case for LLMs is 'summarization'. That is it.

In my experience, bing accuracy mode and phind are very usable, certainly not perfect but mucht better than Bard.

ChatGPT hallucinates a lot, but Bing chat so far for me doesn't do that. And it makes the key parts of the answer clickable to read the pages it got that from. So you can judge whether you trust that page or not.

Maybe, but I feel like an LLM based search has one advantage over Google/Bing as they are today - you can host it locally and keep your searches private.

For non current events I'll take a locally run LLM over privacyless google even if the results aren't quite as good. Google knows this and they are scared.

I think there's very little awareness of the use of cross-encoders for reranking

https://weaviate.io/blog/cross-encoders-as-reranker

but these are significant because they noticeably improve search ranking results and a development like that happens every 5-10 years or so. Getting a bunch of results from Bing and then reranking them with a cross-encoder is a low-hanging fruit in terms of making a better than status quo search engine.

Yeah, that's correct. We also believe that's a good direction for improving search results quality and combining different methods of search: https://qdrant.tech/articles/hybrid-search/

I like that article.

If the cross-encoder is well-calibrated then you really can merge search results from different sources the way IBM Watson did back in the day and not have to calibrate the individual sources.

Yeah good to mention these. I think they may be slightly too slow running for now, as BERT was for Google search when first released, but one day these will help greatly.

Cross-encoders are fine for reranking.

You're certainly not going to run them for every document in the collection but can you run them for the top N? I think the answer to that is yes.

Yes that's true, but performance vs run time isn't an easy win as compared to say using bicoders with pre-embedded retrieval embeddings and just embedding the query (which is almost always a < 0.1s operation even on a low-RAM EC2 instnace).

It takes > 1 second(perhaps 1.5-3s if you embed paragraphs not sentences) for re-ranking k >= 8 on a decent say 32GB CPU instance..(requires encoding each document passage with the search query top k times)... if you have GPUs available for your search inference sure it's just a fraction of a second, but right now I would say it's significantly more onerous than bicoders that get you the first jump in semantic search performance (10-40x slower in practise on CPUs) so the use case differs.

This does not work that well because :

1) Re-ranking is only good as the results you originally get. If you get garbage in, you will likely get garbage out, no matter how you re-rank

2) Re-renking using semantic search does little to help with the main problem in search (SEO spam, low quality content). These 'bad' pages typically rank well in semantic search and re-ranking would probably just help boost them.

A search engine using Bing's index isn't a real competitor to google and bing.

A good example is something like Brave Search, Independent index with independent AI and its own AD network.

But this change really shows that Microsoft believes Bing can take on google now(and it does! BARD is absolute garbage compared to gpt-4), and it doesn't want anything or anyone getting in its way.

By this logic is Brave not a real competitor to Chrome?

The difference is its not relying on licensing Chrome's code, rather its using the open chromium source code, which cab be used regardless if google wants it or not. That's why brave and edge can compete with chrome.

There is a license involved (BSD currently) and there is dependancy on Google moving things forward for Chromium.

If Google stopped developing Chromium or decided to move its Chromium development to a closed-source project, other chromium-based browsers would have hard time keeping up with the necessary development to keep a web rendering engine up to date.

If you want an alternative I've used these guys for stuff:

https://www.valueserp.com/

More like MS-DOS, waiting on the Linux of LLMs

Knowing microsoft has full access to source of GPT fills me with dread.

We need true open source alternatives asap.

If only LLAMA had an actual open license...

As for alternatives, phind is pretty good and uses its own independent model, but it might be killed by this change, as AFAIK, they use bing for results.

Could?

I like how I got downvoted, but looking at MSFTs moves the last few years show a company trying to deleverage its users privacy at any chance, and generally fumbling around like clumsy oafs trying to make new product that will have as much heft as its past OS and Office suites did.

Reply on news.ycombinator.com