Hacker News Reader

Comment by lIIllIIllIIllII | Hacker News Reader

I'm routinely using LLMs in place of search engines. Would you rather type in a cocktail and instantly get a recipe? or do you want to open google, scroll past 50 ads, click through a bunch of blogspam that gives you the history of cocktails, the prohibition, the chemical formula for ethanol and their life story, followed by a recipe that was itself generated by a substandard AI model anyway

edit: I'm prepared for the enshittification of GenAI, but given competitive edge, many people will pay top $ for faster/higher parameter inference, I'm hoping ad support is minimal on the super deluxe god tier AI subscription which I'll have to pile all of my spare cash into.

View on HN

I have two concerns when people tell me they're using this stuff like a search engine:

1. How expensive are these to operate when compared to search engines? If search engines are about as good at information retrieval, but are much cheaper, what does that mean for the future of these services?

2. How good are these systems actually when compared to search engines? I get that for things like code, you can immediately verify results through testing, but GPT was trained on what looks good to humans thanks to HFRL, not necessarily for correctness.

I really wonder what's going to happen with this comparison in the near future. I think there's two possibilities:

1. Running inference on models and keeping the models relevant by introducing new data remains extremely expensive and companies like OpenAI operate at massive losses. Eventually this leads to their products getting worse.

2. Models get cheaper to run and efficient enough to run on mobile hardware without losing much fidelity. OpenAI still might be in trouble in this scenario if "open source" models like Llama2 get popular.

I prefer search engines because I can look through multiple sources. LLMs just give an answer without context or what alternatives might be. They make it difficult to judge to validity of the response.

perplexity.ai usually gives 4-6 sources that you can validate the response with.

As far as I can tell, it performs a search and then feeds the full text of the search results to an LLM and present a summary with citations.

Not all questions need a variety of answers. If its good enough for your needs go for it. Saves some time searching.

> or do you want to open google, scroll past 50 ads, click through a bunch of blogspam that gives you the history of cocktails, the prohibition, the chemical formula for ethanol and their life story

I think in this aspect the difference between GPT vs Google not in level of tech, but because GPT took text from copyright owners and distributes it without their consent and without showing their ads, which is being challenged legally in my understanding.

Doesn't matter. If governments blunder this and try to enforce copyrights in AI, then we are all going to be using Chinese AI in 5 years instead of Google because they don't give a damn about copyright.

CCP already created its own regulation rules for AI.

And what does it say about copyright? Genuinely curious.

internet articles say something about copyright protection, but more important part is that AI has two promote core values of socialism and avoid statement criticizing Chinese government.

Interesting that the CCP rules for AI are the same as the rules for any meatI under their control. At least they are consistent.

It depends on how important it is that I get a precisely correct answer. For a cocktail it's not a big deal if I get one that's slightly different, in other cases an incorrect answer might be catastrophic.

Amusingly, Kevin Roose and Casey Newton made a ChatGPT-authored cocktail recipe on "Hard Fork" a while back: https://www.nytimes.com/2023/06/30/podcasts/hard-fork-ai-poi....

I think the review was basically, "I’ve had worse cocktails. I’ll be honest."

So...meh?

Using bing chat, you always just check the source of its answer if it's crucial. Saves time searching.

If you pay for search you get some of those benefits already

Sincere question: what paid search do you have in mind?

How conceptually could a paid search engine solve the problem of SEO and spam? (Curation seems economically infeasible?)

Is what you're actually proposing ad-free search results? (I'd be willing to pay Google approximately $0.01 per search for this. I suspect this is probably more than they get from anything but the most expensive ads (pharma ads that confused old people will click on).)

What I would really love is a browsing-assistance agent based on something like an LLM that can summarize and de-cruft search results for me and show me content extracted from web pages in a standard format without any Javascript. (I don't need an LLM to be the arbiter of truth, I just want it to be good at summarizing documents.)

(Because Google is so wedded to the ad ecosystem I'm not sure I trust them to be the steward of technology that is basically about removing ads from my life.)

Kagi https://kagi.com/ is the paid search I've tried. It's was decent, but not a noticeable enough improvement to win me over as a customer.

OTOH, I am a paying ChatGPT Plus user and there are some days where I feel like I get my monthly subscription price's worth in a single day.

In terms of summarization, this study Anyscale just published is interesting - llama2-70b gets within a hairsbreadth of gpt-4/human scores for summarization, so it's conceivable that sometime soon, you might be able to run a local LLM on your browser and get decent results (4-bit quants of llama2-70b currently take about 40GB of memory, you need at least 2 x 24GB or 1 x 48GB GPU to run them at reasonable speeds atm): https://www.anyscale.com/blog/llama-2-is-about-as-factually-...

Who do I pay for better search? Honest question, I'd love a recommendation.

https://kagi.com/ is the one I see recommended. The results went to focus hard on "no blogspam" which is nice, and supposedly you can completely ban whole sites from your results forever (e.g. goodbye GeeksForGeeks and your shit SEO hacking)

I like Phind

Or you can run your own open source AI, on your own hardware.

Reply on news.ycombinator.com