Hacker News Reader

Funnily enough, I see this as one of the biggest disadvantages of ChatGPT. I am used to searching the web and navigating results, and have developed a lot of heuristics for assessing the credibility of a page. ChatGPT flattens the true, not-true-but-maybe-helpful, and straight up time-wasting BS into the same confident patter. So I have to treat everything it says with equal scepticism.

I agree. I usually end up finding alternate methods or hints to look for a completely different approach (especially from stack overflow).

The time wasting SEO BS is all but non existent in my use. ChatGPT has nearly eliminated Google because if I want to find the best 10 films involving aliens in the 90s by obscure directors I get exactly that, instead of three million pages of trash.

> the best 10 films involving aliens in the 90s by obscure directors I get exactly that

Actually you're probably getting someone's list of 10 best films that got copied to 1000 more sites and ended up training ChatGPT...

Still counts as a win I guess?

I've asked very complicated contrived combinations that don't have any results on Google. GPT-4 still had no issue assembling the list.

Two Paths:

1. Build a skill set to navigate millions of pages of fluff SEO garbage and gain intuition of which sites are more likely to be credibility based on something akin to a internet dosing rod.

2. Build skill set to evaluate if a statement or claim is factually true. i.e. ChatGPT just told me something, how likely is it to be true.

I know which skill set I would rather develop, obviously #2, as it's transferable to nearly every intellectual venture a human would consider, while #1 is not.

Also, I feel the people that play the FUD campaign on GPT hallucinations just haven't used it much. The average quality of GPT answers, at least in programming, is far above the average hit on google, and the fact that I can interact with the answer and shape it to go in any direction I want or mix it match it with nearly any other concept on the spot, it's light years more valuable than the average google search. At this point, I would guess I only resort to google 10% of what I used to, and it's normally to attempt to find a specific website that has specific information I remember accessing before that I want to look up again.

The real problem is ad business. If chatgpt becomes monopoly we will see many ad banners. After several years of monopoly we will see chatgpt anders lacking so that you will have to see more banners in response. After some time quality of answers will depend on the amount of money you put into your chatgpt subscription. You will not be able to go back to google. There will be no google any more.

The only really workable solution imo is that search companies need to make their money on searches, not ads; something like Kagi I guess. Any perverse incentive is inimical to a really good search engine.

I think the internet is going to start to stratify into paid and "free" tiers going forward, and honestly it's about goddamned time.

A huge drawback of Kagi is that it requires an account and can associate your search history with it. Using other search engines while not logged in is more privacy friendly.

That problem seems very solvable if they don't need to monetize your data. You might, for example, turn off search history and the use of search history to guess what you might mean with new searches.

Turning search history off doesn't mean that they stop recording in in their database.

I have no way to verify that.

Kagi keeps track of all queries because they are charging you for each one or removing from daily quota

Kagi keeps track of query count, not the actual queries.

I really doubt that using Google while not logged in is more 'privacy friendly' than anything else, given their ability to track with pixels and shadow profiles. In any case I don't particularly care about that issue, I'm not searching for bomb-making guides or anything.

What I care about is a search engine that's trying its hardest to serve me answers to my searches, rather than feeding me ads.

Exactly.

AI tools today are great for the same reason Google was great in its first few years of existence: they want to attract users and sell this awesome new product. But this will quickly change.

The ad and investor-fueled enshittification will inevitably also come to popular AI tools, but in even more sinister ways than web search results. Forget banners and traditional ad campaigns. Users will start seeing subtle product placements in the responses, and certain brands placed higher than others, all embedded in the friendly conversational tone they're used to, to make them even more susceptible to manipulation. Thanks to the sea of private data users are willing to share with their "personal" AI assistant, the responses will be perfectly fine-tuned so that ads are targeted at them _personally_. Advertisers will no longer need to target broad cohorts who may or may not be interested in their brand. They will be able to reach _exactly_ people who are most likely to make a purchase, especially if the ad is delivered at _exactly_ the right moment when the person is most vulnerable.

The only way to avoid this is to use open source and self-hosted AI tools, but the vast majority of people will use whatever service is popular and "free", same as today.

Enshittification

https://pluralistic.net/2023/01/21/potemkin-ai/

Yes, who cares if the answer is hallucinated, as long as it's easy to read and stated in clear, confident-sounding language.

I agree in general, but part of what makes ChatGPT great for coding is that I can test the answers. They either work or they don't.

> They either work or they don't.

Or they work for a selection of cases but not others. Or they seem to work in a limited number of tests but aren’t truly doing the right thing. Or they do work but have a security flaw that makes your service vulnerable.

Coding solutions have tradeoffs and subtleties which go beyond “it did the thing right a couple of times when I ran it”. If that’s all it took to verify code is good, there wouldn’t be any bugs; you can test code a person writes too.

Yes, ChatGPT can't do all the work for you. You have to know how to test the code thoroughly and think of edge cases and performance tradeoffs and all those other subtleties.

But contrast this with other types of information. If I ask ChatGPT "how hot is the surface of the sun?" and it answers "15 million degrees Fahrenheit", I have zero ability to verify that information. As a result, I don't find ChatGPT useful for most informational searches, except the ones related to coding.

(The example answer is wrong according to https://www.space.com/17137-how-hot-is-the-sun.html, albeit in a way that I could see potentially confusing an LLM.)

It's not much worse than a random blog or a stackoverflow answer. They never give you a guarantee that their answer is right for you.

But the nice thing about ChatGPT is that you can give it more context than it is possible in a Google query, and you can ask follow-up questions.

Yeah, in my experience, Stack overflow answers suffer from subtle errors all the time too. Especially for basic questions -- the people answering and upvoting the answers are inexperienced devs, so they often upvote answers that are either the wrong way to do it (eg. reimplement something that has a very elegant built in way to do), or they just contain bugs (eg. because they mixed up the number of characters vs the number of bytes in a string).

They are still useful, you just have to make sure you understand what you are doing and read the relevant docs before you paste the code.

Actual code, yeah, ChatGPT probably will mess up.

Things like "How do I delete a value from a dict" probably either works or it doesn't.

Then again, for that level of simple stuff, one can probably search a PDF cheat sheet just with their eyes faster than they can type the question for ChatGPT

It might sound weird, but you can actually ask it to do this too.

On the surface it makes no sense to do this since it already gave you an answer. But that's not how LLMs work - they can simulate a variety of different "humans" depending on the prompt and presenting a different prompt persona (security researcher, code reviewer that focuses on edge cases, etc) yields a different quality of response.

Well, but you still don't know if any of those persona are correct. It's great that you can talk to it, but ultimately you have to make the final judgement call, as it will bullshit you, and it can be much better at bullshitting than most humans.

So basically the same as with an answer from Stackoverflow but without the user voting.

It’s not really an issue for me because I generally know very quickly if it’s wrong or hallucinating and can easily get it back on track.

And in general I find it’s more accurate and far more concise than anything that’s going to show up in a Google search.

I mean, there’s no way I’m taking what it says at face value or copy / pasting code from its response, but that goes for stack overflow, or any other source too. (maybe sans official docs)

I'm curious. How do you know it is speaking untruths? What kind of questions do you ask where you know what a wrong answer would be? Is this just like "remind me how to reverse a string in go" kind of thing? Where you know how to do it, but cannot quickly recall?

Yep, exactly. I ask things that I know the answer to, or at least know enough to understand what the right answer is going to look like. I just can’t recall the specifics or am busy thinking through something else. It’s basically just an assistant, saving me the time of digging through official docs, searching previous project code, my own notes, etc.

I mean, it works for politicians.

There’s no guarantee that the SEO websites answer isn’t hot garbage too. You’re rolling that dice either way

It used to be why Recipe websites were so long, but that’s no longer the case.

There’s three reasons…(1) ad revenue, (2) there’s a belief that the extended life stories confer copyright to recipes, which otherwise cannot be copyrighted and (3) everyone does it.

Source: I made SimplifyRecipe.com, which requires diving into the minutiae of recipe authors.

Otherwise, I agree that getting consistent results from GPT is super useful.

I've been doing the same

For basic stuff it is much faster than Google. But if it requires minimum thinking it still faster doing myself.

The other day I was lazy to find a generic scraper I have so I could save a reference manual. I know it is just like 50 lines of code and if you are smart you could do it async. I tried with ChatGPT and it didn't work at all. Many silly mistakes hidden between comments that looked reasonable

I love the way there are 10 competing folk explanations for "why are recipe pages online so long?"

Spending time on the page should actually be a great proxy for usefulness on recipe websites. If it's a long page of introduction and anecdotes, maybe it took me three minutes instead of one to reject it as unpromising. If it's the recipe I actually used, however, that tab is going to be open for the next thirty minutes or more.

Are there really 10 different explanations?

I’ve seen “they’re long so there’s more space for ads” and “they’re long so you spend more time there so it doesn’t get penalised by google”, and to me those are pretty closely related.

There's also the "recipes aren't copyrightable, but the fluff about grandma baking cookies is" explanation.

I see that one a lot and it doesn't hold up to scrutiny because copyright on grandma isn't how recipe websites make their money. They trade in a generic field flooded with fly-by-nights that exist exclusively to run ads. They don't care nor need to care whether their filler material isn't copied.

I've also seen "the recipe itself isn't copyrightable, so they add prose/filler that is".

But why would staying long on a web site correctly indicate the page provided the answer? Is you spent a short amount it could also indicate you found what you were looking for, and off you went.

That's the neat part: it doesn't!

It's a situation where Google doesn't benefit much more from it being the actual answer, and the site doesn't either.

In the long run it could have an impact on the user perception, that could be nullified by the page still staying ranked higher than the competiting pages and the user finding nowhere else better.

And how would google know if you didn't make another request

Google Analytics or Google Chrome telemetry.

It happened when Google Penguin was rolled out, because Penguin’s claimed focus was on better content itself to combat backlink spam practices that abused PageRank descendants. It sort of worked but there were entire valuable websites with great content (but according to Penguin comprised of non-rich content) that were dropped pages deep into the search results. Recipe websites were among these. There exist reams of articles on how to fix your content to get Penguin to like it, and the buried recipe nonsense started exactly at the same time.

https://en.m.wikipedia.org/wiki/Google_Penguin

Can’t wait until they (actually) monetize ChatGPT and maximize the time spent on the website, like Google. Can’t wait until ChatGPT says that, when their grandma went to school, they too needed to mentally add 26 + 13, and the teacher would always frown on pupils when they gave the answer 31.

ChatGPT’s strength is that it’s new on the market, needs to build an audience, and therefore their UI and UX are lean, devoid of gaming.

Same as Google, up until 2008.

I'm not convinced this is the case. Unlike Google, ChatGPT has clear monetization channels which are not advertising, like ChatGPT+, which is already doing very well, and it's easy to imagine further channels which allow access to specialized models. Additionally, unlike Google, you can only really get away with making your service crappy when you have such a strong moat that it's still significantly better than the competition. The only moat OpenAI has is its model, which is proving a lot easier to replicate than Google circa 2008's moat.

Cable TV has a clear monetization channel, surely there are no ads there.

They have. We pay $20 per month to get priority access.

A lot more than I've ever paid to use the Google search engine.

Paid for service over ads is the new way forward.

Ads and shit doesn't exist because there isn't a clear way to extract value. They exist because the folks that make decisions love ads and shit. You will end up paying a subscription for a bad product with few alternatives, just like most folks end up using Google now and pay with all their data. There simply isn't a good search engine, Google murdered all the real competition and what was left is mediocre.

ChatGPT now is like Google in its beginning, sooner or later ads and "optimizations" will kill it.

There is a YouTube Premium and I don't see a reason why there couldn't exist Google Search Premium but that again doesn't solve the problem of SEO spam and garbage websites.

> I don't see a reason why there couldn't exist Google Search Premium

Probably because there would be barely anything left in the results once ads and spam are removed.

It would make room for relevant results.

Especially for these type of simple syntax questions that autocomplete or copilot isn’t getting. It’s so so quick to ask ChatGPT for the answer, rather than finding the documentation for the thing you’re looking for and parsing it.

Definitely the fact that I am not going to deal with ads when I am looking for something is a gamechanger.

I do wonder: does / how does chatGPT differentiate between ad-driven content and regular content?

For example, if I ask what the best vacuum cleaner on the market is, I'd rather it not regurgitate bestvacuumreviews.net which is affiliate (not quality) driven

even the fact that you always go to that same window, with the same format and interface lowers cognitive load imo. Of course chatgpt hasnt been a magic bullet for me yet, but it definitely eases the pain of developing things outside your usual perview

I’m travelling in France at the moment and chatgpt is an absolutely incredible tour guide.

Yesterday we asked it what “RF” inscribed in a crest on a government building meant (“République Française”). A few days ago we asked why an area was called Gobelins, and if that had any relationship to the modern goblins in fantasy. I asked how to ask for more bread at the table and it gave me a pretty long, fancy French sentence. I asked it to simplify the sentence and it did beautifully. You can ask Google translate to translate something, but chatgpt can give you multiple translations, and tell you the cultural nuances of all of them and teach you French grammar while it’s at it. “How do I say this? Why does it use suis in that sentence? Is that a weird thing to say to the wait staff at a restaurant? What is the protocol there?” Etc.

We’ve been using it constantly.

And, not only that. When it explains something and you don't understand, you can ask it to rephrase or give analogies. I guess the only thing missing is imagery

Think bigger than imagery. Just have it create the physical thing you seek.