Hacker News Reader

Comment by culopatin | Hacker News Reader

I feel like I’m missing out on something but other than to summarize text or fill pointless and lazy homework assignments I don’t have much of a use of gpt search. I think I have a trust issue with it and I rather get the raw result and process it myself than to believe that what this model “understood” is right. Anyone else feels this way?

View on HN

Here is what I have asked it the past few days with great results:

  - What is the trim stop attachment that comes with the Festool DF 500 used for?

  - What are some options for water for a home with no municiple water or well access?

  - What dimensions drawer should I make when using 18" Blum Blumotion full extension drawer slides if the drawer opening is 20 inches wide, 18 inches deep and seven inches tall?

  - Can the smaller Laguna 14 bandsaw motor be replaced with the 3HP version?

  - What's the best way to get ChatGPT support if the official support page is not working?

  - I'm trying to remember a 90s movie about a boy prodigy that goes to an event with other prodigies and then shouts out an answer to a math question from the audience.

There's a ditch by my house that the city put in. I roughly knew what it was, a big pit where rain collects and soaks into the groundwater. But I wanted to know the name of it and no amount of Googling could get me to the right answer. My query was too vague.

The day that ChatGPT launched, I described the what it was and it came back with, "that's called an infiltration basin."

That's when I knew Google was in trouble.

You have touched on something I am incredibly excited about. ChatGPT is going to do away with a lot of the intro level questions you ask researching certain sorts of technical problems, especially as it becomes better able to understand images. ChatGPT removes the need for you to ever open a manual (unless you want a deep understanding).

How do I turn off this feature in my car?

How do I replace the fan in my fridge’s compressor?

Why is windows repair not working in this scenario with this PC?

This tool is going to be incredibly deflationary in many services geared around repair when now it can tell you almost exactly what to do and soon it’ll be able to produce videos or images of each step on demand.

> and soon it’ll be able to produce videos or images of each step on demand

As long as they're not hallucinated media...

Most of the Googles SERP results already are delusional or hallucinating spam, so the bar is low and for me at least ChatGTP is less bullshit.

Yeah, when the “site:” search hack doesn’t get me an answer and I fall back on the normal SERPs, most of the time the top several results are all:

“Authoritative title that references my exact search” -> page body restates my question, gives an oddly reworded summary of Thing in Question #1 -> “some people online have said” followed by snippets lifted from the ether -> repeat for Things 2,3,etc -> “our suggestion is to look at the features and decide based on what other customers/users are saying”

It’s perfected vapidity wearing the skin of an editorial review site and takes a LOT more cognitive load to suss out the phoneyness compared to the last time the SERPs were packed with spam (~10 years back).

hallucinations don't invalidate the product itself. I really hate to see this constant lazy refrain on here.

Its just like anything else. Trust but verify. Some will just trust, others never will... People probably over-trust the tech now, but just like wikipedia it will be approached a little more carefully as people learn its shortcomings.

Unless you're trying to perform open heart surgery or doing something intrinsically dangerous, its probably going to be additive. I wouldn't trust it to help me assemble a warp engine, but it may be able to give me some decent pointers for guitar technique or how to change my oil.

It’s not a lazy refrain, it’s a serious downside of the technology. It’s tiring and stressful to supervise the work product of an assistant that is extremely capable sometimes, but a compulsive bullshitter at other times. Just as it’s tiring to supervise an “autonomous” car that often deftly navigates the road, but sometimes wants to plow into a school bus.

I think that the concern about LLMs making up information are absolutely valid, and a serious concern. At the same time, I think most people focusing on those concerns aren't thinking much about the alternatives, namely Google.

For many people, Google results have gone from being an absolutely amazing demonstration of what could be done if one had the resources to crawl and process the entire internet to a complete waste of time at best, clogged with spam and nonsense and misinformation. I've been using Duck Duck Go for years now, and recently my middle-schooler switched to Duck Duck Go because I got answer to a question she had already asked Google before checking with me. Some people even pay for Kagi search, because Google is just that bad.

So LLMs don't have to be perfect to be better than the alternatives, not even close.

> Some people even pay for Kagi search, because Google is just that bad.

Small correction - from what I have heard in our community, most of our users pay for Kagi search, not becuase Google is just that bad, but because Kagi is just so good. In other words we are making the value proposition be "best search in the world", one worth paying for.

(Kagi founder here)

> - I'm trying to remember a 90s movie about a boy prodigy that goes to an event with other prodigies and then shouts out an answer to a math question from the audience.

I love that movie! I had the same experience a few years ago of trying to remember its name.

This is a great example of something ChatGPT does amazingly well at. I did almost the same thing myself recently, trying to find the name of a show featuring an actor in a different show or something. A straight-up search returns nothing for it; you have to look up the actor's name, then look at his shows, then try to remember the name. But a search with ChatGPT found the result easily.

I tried a "similar" thing now. I was thinking of the movie Gifted, and wanted to see if I could get ChatGPT to pick it up, here's my query:

> I'm trying to remember the name of a movie about a child, featuring an actor that was in a marvel movie.

And the result:

> There are many movies featuring actors who have appeared in Marvel films, so it's a bit difficult to pinpoint the exact movie you're thinking of without more information. However, one possibility could be "Gifted" (2017), starring Chris Evans, who is well-known for playing Captain America in the Marvel Cinematic Universe. In "Gifted," Evans plays the role of a man raising his young niece, who turns out to be a child prodigy. If you could provide more details about the plot or other actors, I'd be happy to help narrow down the list further.

This is obviously a tiny use case, and really not worth thinking about. Who cares, right? I thought the same thing about search in general 20 years ago, and that turned into one of the biggest businesses of all time.

This is the search that I did that made me the most afraid for Google.

This is hilarious- now when I google "90s movie about a boy prodigy that goes to an event with other prodigies and then shouts out an answer to a math question from the audience" the top result is the IMDb page for Gifted!!

The actual movie is Little Man Tate, yes? I've never seen it but that's what ChatGPT tells me. Zero Google results point there, but it is the #5 result for "related searches." I hate Google and even I am shocked how bad this is.

Do you mind providing correct answers for those? (and did GPT get them right?)

What's the best way to get ChatGPT support if the official support page is not working?

Ooh! What was the answer to this one?

I tried to sign up for ChatGPT, but the verification text never arrives.

(No, it's not a soft phone. I've had the same phone number on the same cell carrier for over 20 years.)

It said to email support@openai.com

To be fair, Google is usually pretty good about answering movie questions. It guessed Gifted in your case, is that right?

I agree with your sentiment exactly. But your comment made me think about this an interesting way that I hadn’t thought about before, that made me better understand why I am “anti-GPT search”.

All the same things that the model makes an effort to “understand” are things that can contribute to your own understanding. When it “understands”, you do not. You get the summary and miss out on a great amount of nuance that can come with learning and finding the right answer for yourself.

For instance, it is very common that in the search for why X is happening in my code, I will find tons of information that don’t answer my question but help me form an understanding of why it is happening, how the system is working, etc.

This is especially visible in science. Many papers outright conflict with each other. Some have better or worse methodologies than others. Some have better analyses. Some state outright falsehoods without citation or misinterpret citations. Having a GPT “understand” this to produce understanding in humans is going to end very badly. It takes several hours for a human (in my experience) to understand even a single academic paper on its own.

I suppose the appropriate cliche is “it’s the journey, not the destination”.

A great quote from an article I read on HN recently

> Suppose I was an evil person and wanted to eliminate the curiosity of children. Give the kid a diet of Google, and pretty soon the child learns that every question he has is answered instantly. The coolest thing about being human is to learn, but you don’t learn things by looking it up; you learn by figuring it out.

As opposed to the thing we did before, which is when kids asked something they were told something that's wrong and told to "stop talking back" and "go to your room". I think this is still an improvement.

> As opposed to the thing we did before, which is when kids asked something they were told something that's wrong and told to "stop talking back" and "go to your room"

Not everyone had your childhood.

Not everyone, but lots of people.

This is silly. You could extrapolate the same argument to use against Libraries.

Most of the people here know and understand computers because we had to tear everything apart and figure out how everything worked just so we could play some video games.

Contrast kids of today who can just go and play video games, no disecting of computers required. Naturally, most of them don't understand or appreciate computers as anything more than just another appliance.

Not silly, it's a quantity into quality thing.

Do you have concrete examples of stuff you figure out instead of looking it up? I'm not going to figure out the population of the Liechtenstein, I'm going to look it up.

What chatGPT misses is I want many sources of information conflicting or not so I can form a more informed opinion. I want to know every side not the correct one.

The most illuminating thing about ChatGPT for me is just how terrible most programmers on HN apparently are. I thought it was just a funny meme that all we do is copy/paste from Stack Overflow but apparently that is literally what a lot of people are doing all day.

People who blindly copy from ChatGPT or Stack Overflow are most likely either very inexperienced or simply bad programmers. However, from where I see it, ChatGPT, Copilot, and any other such tools are fantastic at prompting you to think differently, getting you past writer's block, giving you ideas, or saving you some time googling for syntax yet again. You need a nudge to trigger recall of what you already know. It's a fantastic tool, just like an IDE.

Meh.

I was working on a personal project yesterday to answer some questions I had about how liquidity risk works for money market mutual funds, and to forecast/nowcast liquidity risk and NAV risk for a bunch of funds.

Mind you: I don't know the first thing about anything financial. I was just curious.

chatgpt gave me a bunch of sources of data that I wanted, translating my lay description of things I wanted to know into financial terms of art. I could then look up legal definitions and formulas for those terms to make sure they were what I thought they were. chatgpt also told me which SEC forms those things are disclosed on, what data brokers I could use for other data, etc.

between chatgpt and copilot I saved at least an hour on the job of pulling down historical data from EDGAR for a bunch of funds and getting the stats I wanted (I didn't know EDGAR existed until yesterday, and the xml/html/txt formats are kind of annoying... like, fine, but a bit of a pita so I'm glad I had help because ughhh is that kind of code boring and damn are LLMs good few-shot inductive parser generators!). Also wrote some nice chart.js code for me and helped with automatically collecting, searching, and extracting some key stats and terms from prospectuses. I didn't know about chart.js until yesterday.

All of this would've been possible without assistants, and required a lot of "executive function" on my part to bring together, but it seriously saved me at least a couple hours of implementation work and up to a day on research and learning terminology and regulatory stuff. Again, verification of those things is way easier when you know what words to look up definitions for. And chatgpt did make mistakes/hallucinate.

I don't find much use in my professional life, where the code I'm writing is apparently too domain-specific for copilot to be helpful and the mathematics is too complex for chatgpt to help with. Maybe in a few years. We'll see.

It's sad how most programmers are so terrible they need IDEs to automatically create stubs for required methods or getters and setters. They should have to type all of that out every time, and from memory, too! VIM with no plugins is the only way to do it!

Or, you know, smart developers use tools appropriate to the work, and some of us have figured out how to use this new tool before you have. That's okay, you can catch up!

This comment is giving me “Look at me I’m so very smart” vibes.

If you don’t have enough insight to understand how GPT4 could be useful for engineers you’re not as enlightened as you think you are.

This comment is giving me “Look at me I’m so very smart” vibes.

I don't think I'm so smart, what I am saying is the GP is being extremely dismissive of other engineers here on Hackernews. Every post about LLMs is full of engineers stating how these have been useful to them. GP simply asserts that these are all "terrible engineers." Do we think that's REALLY the case or in fact LLMs can be very useful, especially when for example learning a new language or framework they haven't used before.

Example: recently I was curious about cross-platform UI development so I used ChatGPT for research into different frameworks and made several apps in Java and .Net. Not being an engineer who's built many GUIs before, ChatGPT was amazingly useful for this and help speed my progress. Does this make me a terrible engineer?

some engineers != all engineers, that's classic unrepresentative sample. nice it helps you, hope you won't forget how to code in future, or you will be handicapped in case of openapi outage.

Did you suddenly become defensive?

For me it’s not even theoretical. I’ve already wasted my time trying to sus out subtle garbage mixed into its answers. It’s the same reason I stopped using copilot. I don’t need or want that.

Funny, so far i've been just asking it things i normally ask Google. Same paranoia of answers that StackOverflow/etc give me, but far quicker and more responsive.

TBH i'm not sure under what scenarios people search into Google expecting perfect answers. Ie where we're disappointed by incorrect information. I filter through a dozen wrong answers on Google every day. Why is it different if it's from ChatGPT?

I guess one difference is that I have a lot (decades?) of experience in the types of errors humans tend to make. Humans have certain biases, certain blindspots, etc. In programming for sure, there are very strong patterns in the types of mistakes we make that tend to be a function of skill or experience. So you (I) look for those without even thinking about it. Violation of YAGNI, ACID, DRY, pre-mature optimization, etc.

Even before getting to an answer I can often judge how trustworthy or skilled the author/speaker is likely to be by all kinds of little signals or keywords in their speech.

ChatGPT (and Github co-pilot) disables that highly tuned error detection and correction experience. It makes unpredictable mistakes, which is peculiarly pernicious. I'm sure it varies by the temperament of the user. Personally I don't have the patience for that.

Good points. I think it also helps that i purposefully avoid asking for things that i can't validate. The easier it is to validate the more likely i am to ask.

I find ChatGPT especially great with giving starting points. It kinda feels like 10 (or something... time?) years ago when i discovered how good Google was at finding movies based on vague wishy-washy definitions. If i don't know what to call something ChatGPT does a better job at pointing me in the right direction. Often giving the right answer.

But i'm not saying it has lasting power. We'll see. So far i'm using Search engines less than ever.

On further reflection one of the major missing signals from ChatGPT is time. This is especially important for technology. When someone's explaining, giving an opinion, providing a solution -- I always consider that in the context of the date attached. ChatGPT gives us these answers that remove that.

I see completely the difference in your example, though, where you can easily and immediately validate the response. It definitely is way stronger at that than previous solutions.

I certainly have the trust issues for things where correctness matters. For things were it doesn't and I already know how to do something but don't have it memorized, I think it is very very convenient.

I use it various times a week for cooking. E.g. "basic recipe bechamel sauce in metric"

ChatGPT: Within a couple seconds I get the ingredients in metric and a step by step guide.

Google: The provided summaries are useless to me since they are not in metric. So I have to click on a link and then start scrolling around all the noise of history of bechamel sauce, anecdotes about someones grandparents, pretty pictures, etc to find the actual information. Without ad blocker it would be an even bigger nightmare.

Before a few keywords and quick website scan.

Now you have to engage in a conversation and get the 'a feel lucky' version where before you got to look through different more visual recipes

I don't need to engage in a conversation. Query was exactly the same for both.

There is a difference whether I'm looking for interesting, more complicated dishes I haven't done before, or fairly standardized, simple recipes I have done many times and remember the technique but not things like ratios.

For the former I use a select few recipe sites or youtube channels directly. Google is miserable for discovery in my opinion. For the latter I use either my own recipe notebook or more recently ChatGPT as well.

Yup, and maybe we're just getting old. There are still people on HN complaining about Google removing the OR and AND modifiers, meanwhile the younger generations write queries like "where should I eat breakfast today"

Those aren't really exclusive. Different problems, different query types. (I wouldn't quite do the second example like that, but "where can I get food here" is fundamentally different problem than "I hope there is somewhere out there a bootleg copy of the datasheet for this very weird part and no I don't mean any of these 5 easily-confused other things", which is when the lack/ignoring of detailed modifiers drives you up the wall)

I'm just trying to point out that that query would have returned nothing much at all back in the age of keyword-lookup search.

I'm fine with writing queries like "Where should I eat breakfast"

But I hate seeing the top half of the screen are ads for huge chains, and then the top results are blog posts about "Where are the top 10 places to eat in your city" and not a list of restaurant web pages.

I used to try to be surgical with my queries but I've given up and just type a question at my search engine because if that's what they want as a query well they're going to get it.

Not since Google results became such garbage. Plus, the chats provide references. So easy to verify.

I love that Bing Chat actually links to references. It felt like something was missing from ChatGPT without it.

If I have to check each reference, what's the added value compared to just giving me a list of links (i.e. a good old search engine)?

Chat search tools are only useful if (when?) they achieve a 99.99% reliability (I'm fine with one mistake per 1000 searches).

Just because it does, it doesn’t mean they’re being interpreted correctly, so watch out for https://xkcd.com/906/

In the context of LLMs they aren't really sources, just additional reference material. I'd trust what Bing Chat tells me far before I'd trust Google's new Bard AI. It legit started making up non-existent commands for me to use.

I find it useful in Edge where it's aware of the tab contents. It's nice to be able to ask it to summarize a long article, or find Reddit comments about it and summarize those. I think it uses Bing's search index because it does not seem to be aware of my personal data on my tabs

I thought the same but it is actually able to interpret URLs that are not indexed by Bing. It doesn't work all the time though, don't know why.

Is that an opt-in option or a default 'feature'?

I've found that if you push through the trust issue with the expectation that it's not a final answer, it's just getting you closer to it a lot more quickly, then that ends up being very helpful in a lot of cases.

Well, Microsoft forced me to use Edge to open Bing chat, so I did that and asked it "how can you use bing chat on chrome?". It pointed me to a chrome extension that allows that. See, it's useful.

Do you have Plus? chatGPT was annoying when I was using the free version - I would have to keep logging in again and again and the service was unreliable.

Ever since I switched the Plus, it's (usually) always on and doesn't log me out.

Makes it much easier to use as a general search engine

I don’t spend that much time in it to even consider that, and so little time that signing in again is not a problem for me. I guess I don’t know what to ask of it.

A director at my company is feeding sparse conference call notes to chatgpt to get coherent written summaries

Reply on news.ycombinator.com