Tell HN: ChatGPT has become untrustworthy | Hacker News Reader

Tell HN: ChatGPT has become untrustworthy

14 pointsgtirloni15 comments

This has happened for several weeks now with GPT-4:

I ask it how I can do something that I know is possible and just want it to get me the right syntax. It will say "X can't be done with Y but you can use Z, here's how".

I know it's possible so I just say to it "I can't believe doing X with Y is not possible" and it immediatelly says "yes, doing X with Y is possible here's now"... and now I don't know if it's just "complying" or if it's really possible or not (maybe something changed and it's not possible anymore?).

So now I have to go to Google/Kagi and search it myself to confirm it's giving me the right-ish answer and not just playing nice and hallucinating.

I've cancelled my ChatGPT subscription today until/if they can ever get it back to how it was several months ago.

ps.: Claude 3 Opus doesn't seem to suffer from the same issues.

View on HN

All AI is untrustworthy. Many people have been saying this since the start, and there's not really any evidence that AI will or can attain perfection. Set your expectations accordingly.

P.S.: There is no greener pasture. If you've been following the scene since the BERT/GPT-2 days, it becomes readily apparent how "far" text generators have come. They got bigger, more expensive, slower, censored, and 10-20% better at solving human reasoning problems. There is no light at the tunnel suggesting infallible LLMs are coming. You are being sold a moving goalpost.

Not true. Things have degraded considerably for ChatGPT.

Yes, it's just a stochastic mechanism but there are degrees of accuracy and they have gone down lately.

If you set your expectations accordingly, I find ChatGPT answers fairly helpful. At least for my experience.

I actually tend to agree. My struggle is with people that sensationalize the untrustworthiness of AI when it has been plainly apparent since day-zero.

I have experienced such issues with GPT-4 returning the wrong syntax, especially when asking questions about complex operations in Polars.

We are actually running performance monitoring for GPT-3.5/4 and Claude-2 to systematically track such regressions: https://demo.uptrain.ai/benchmark/. We would be curious to know your thoughts and feedback.

I'm not sure if I conciously noticed it but I did notice today that I just haven't been using ChatGPT or LLMs for any programming tasks for the past two weeks or so. I think it has been giving progressively worse answers or I've just hit the limits of how useful I find it. Claude isn't available in my region but maybe it's worth a try.

You can try Claude 3 through Poe.

Use an LLM that searches for answers online, like Bing Copilot or Google Gemini.

All LLMs are inherently untrustworthy, they can be great to get the ball rolling so to speak but you have to double check everything they generate which is its own kind of overhead.

Of course, because it was inherently trustworthy before. Goddamn, we're all lost, aren't we? Either that or you're really into believing the hype.

You completely misunderstand LLMs if you think they ever were "reliable" in the first place. All it is, is a plausible text autocompleter.

In other news, Google returns spam sites and Excel parses dates incorrectly.

How is this ramble of a post became front page on HN is beyond me.

I mean for anything you could have easily verified via a search engine, I would say you always should have been verifying. I've had ChatGPT give me incorrect but correct-seeming information many times, and not just in the last several weeks.

It's a work in progress and it still has to be done mostly by humans. Humans make mistakes.

Claude is in many ways superior to ChatGPT, reverse is also true.

I was testing ChatGPT's ability to access current events and asked it if it knows about the "bloodbath" controversy. Just that word. It briefly described the mainstream media's version of events. I asked it a few different ways if it was aware of another interpretation, it was not.

Its media bubble appears to be robust. I suppose if you trust the mainstream media that's a feature.

Reply on news.ycombinator.com