Hacker News Reader

Comment by theossuary | Hacker News Reader

This interesting, I've had the opposite experience so far. I tried to do that because if it worked it'd be game changing; but it seems every question I ask it, I get half truths and lies back. Its impossible to figure out what is and isn't true either, because the LLM says everything with such a certainty. I end up having to research everything it says again to figure out what to trust. Sometimes it's good to get a foothold in a subject, but that's all I've been successful with so far.

View on HN

You have such an experience with technical problems?

It’s written at least 20 python scripts with me and almost all of them have been close to perfect almost on first draft.

Really? That hasn't been my experience. Out of 20 python scripts, at least 50% have some error, some of which look right until you run them. A few involve hallucinations, like calling non-existent methods.

Actually yes that’s true, but within a few iterations it quickly repairs those flaws when given the errors and changes. I’m always running the draft script immediately and iterating on it. Sometimes in same context but often I’ll take it’s script, change it a bit, then start a new context with that version to keep adding features.

Are you using 3.5 or 4? And what kind of things are you trying to do ? It's not like asking it, copy + pasting results + paste errors loop takes a long time to validate if it works or not. Even when it doesn't work it saves me time vs google-ing and also not solving my problem. Or gives me insights.

In all honesty I've only been using InstantClaude so far due to the larger context window being something I'd want down the line. What should I be using in your opinion to get the best results?

Examples I can think of where it gave wrong info was:

- I asked to compare the size of the MacBook m1 and the 16" framework laptops

- I asked how to add the nixpgs-unstable channel in NixOS.

- I asked to help write a power shell script to create AD accounts.

I don't know, I've only used chatgtp 3.5, haven't tried other things. Out of your questions I would think only the last one would be a good fit.

The first one is open ended and I don't think M1 came before the "cut date of knowledge" in chatgtp. The 2nd one is too niche, likely not enough data to grab.

For the 3rd one, I don't know if the "date of knowledge cut" isn't recent enough - powershell became a thingy more recently and isn't wide adopted.. but I would expect to be a good fit

Powershell came out 2006, well before the knowledge cutoff date of 2022.

Try ChatGPT (GPT-4) or at least Claude 2.

Claude Instant will not give you the highest quality answers.

So shouldn't you try gpt4 before complaining?

It's incredible to me that so many people still don't realize that GPT-4 is a completely different level from other LLMs.

It's like if an alien visited and the first place they went was to a convention for Special Needs people and then they decided that humans were too stupid to talk to. Or rather, there was a Mensa convention next door with a $20 entrance fee and they figured it wasn't worth it.

Or rather, for $20 everyone at the Mensa convention agreed to log in to a special app and wait 24/7 for any time you have a question. Then they would instantly select the most knowledgeable among them in that subject and immediately enter a reply incredibly quickly.

It "proved" that P = NP the other day, for me. So I think the Special Needs people would still be ahead.

Reply on news.ycombinator.com