Hacker News Reader

Comment by dom96 | Hacker News Reader

I've had a play with ChatGPT and the experience has been pretty frustrating. It either responds with "Sorry, I cannot do this because I don't have access to the internet" (even if I am giving it prompts that don't require this) or it actually generates code but it's subtly incorrect (this was the case when I asked it to generate an example of how to render a 3D cube in JavaScript).

This makes me wonder how much time people are spending optimising the prompt to get the answer they want and they just make it seem like this was the first response they got.

View on HN

I'm pretty confused trying to connect all the reports online with my own experiences as well. From what I've tried, ChatGPT does not _understand_ code at all, and there are many inconsistencies in what it says. The "confidently giving a wrong answer" problem is very real, even if the answer might look very correct at first sight. This holds across all the topics I've tried.

When people say they implement complex tasks with ChatGPT, I have to assume that it's a highly iterative process and/or that they are doing part of the design/problem solving themselves because even for a simple task I could not rely only on the bot's reasoning. (Maybe it gets things right in one shot sometimes - but my sense is that "on average" that's not the case at all.)

All that said - the progress here is really impressive, and I'm still having a hard time wrapping my head around what this can mean for the future.

Confirmation bias - people want it to be a silver bullet so that they can make a blog post about how ChatGPT is amazing.

Exactly. As the OP of the blog, the amount of handholding I had to do for it to understand the syntax of an extremely tiny language was a lot. On the other hand, I’ve messed around with codex and other models before, and something about explaining in normal English, as though I was having a conversation rather than just listing some commands made it much easier. I’m excited not because of what exists right now, but because this shows so much promise even just 1 or 2 papers down the line :D

What a time to be alive!

Seems most likely that you're not the chosen one to hype this new shiny trendy thing, so it doesn't waste precious CPU cycles on you.

It is only if you have truly, zealously dedicated your life to promote ChatGPT in mainstream IT circles, as in getting paid to do so, only then will it completely unleash its vast potential into the reply form, writing you a desktop OS in Brainfuck that is ready to compete with Linux, OSX and Windows, proving the Fundamental Theorem of Algebra, simulating 2^1024 qubit machine that cracks 4096 bit RSA, finding out 23 hidden bugs in x86 microcode, telling you which gene to edit to get rid of peanut allergy, etc etc etc, all at your correctly formulated finger snap.

Full disclosure: this reply was generated with ChatGPT.

Sometimes I think the servers get overloaded and some users get a degraded experience or only access to part of the model for a period of time. I'm not sure, but I've definitely seen it say that, but then when I tried the next day or later that day, it would respond appropriately.

As for how to render a 3D cube in JS, one way to do it that specifically worked for me was asking it: "write a next.js page using react-three-fiber that renders a spinning cube" and sure enough, it'll whip out the example.

May work for vanilla js prompts too, haven't tried. But if you mention the specific library three.js it'll probably respond better.

Instead of optimizing the original prompt, if it spits out something wrong, try and point out the mistake, it is pretty quick to fix it in most cases.

The problem I had was that I didn't see where the mistake was... once I know the mistake pointing it out is pointless.

Feeding back the error you're getting, or the way in which expected behavior is different from observed, can get you pretty far. The bot is fairly graceful at taking feedback. (Your mileage may vary - it works sometimes, but not always. I've also had the bot say "Ah your error was actually <something different than my error>, here is the solution".)

I had an interesting interaction where it said something wrong - I corrected it, and it accepted the correction. I was then curious to what extent it was a pushover - and took back my correction and said that what it originally said was right. It then responded along the lines of "I'm sorry for causing confusion - but <correct statement> is right, and my initial statement was wrong". Pretty impressive!

You summarized modern AI : good for cherry-picked demos, not reliable enough for the real world.

We need more fondamental research to break that barrier.

Depends on your definition of "the real world". The hardest real world problems are out of reach (and always will be, because we'll keep moving the goalposts), but it's already capable of handling easy real world problems, and we have quite a lot of those.

For example, it can answer homework problems and even help design lesson plans, but it can't design a lesson plan that resists ChatGPT-based cheating:

https://alexshroyer.com/posts/2022-12-04-Hello-ChatGPT.html

You can ask it to generate a prompt that when given as an input to GPT will produce the thing you want. In a separate tab run the prompt and give feedback to prompt generating tab.

> This makes me wonder how much time people are spending optimising the prompt to get the answer they want and they just make it seem like this was the first response they got.

That's sometimes true, but much rarer these days.

Rather, think of it like any skill you have to learn. Someone who doesn't understand much about programming could watch a video of a good programmer writing some code very quickly with awe, and assume the video is a trick in some way. But it's not - the programmer just has enough knowledge and experience that she can do things that other people can't, and do it quickly.

Similarly, if you spent a bit of time working with and learning how to use these models, you can get crazy impressive results every time. You don't have to cherry pick much or at all any more - you just know how to use it properly.

Generally longer inputs can help reduce the amount of cherry picking you need. And of course there are many jailbreaks to get around no access to the internet. In this demo I actually didn’t use any! :>)

it revived my imposter syndrom because I see all the cool tricks people come up with naturally while I get half of what you describe and half easy naive answers :)

While what’s on the blog isn’t cherry picked, it often requires way more context than a human would to solve a problem. For instance I omitted the 100+ message back and forth where I explained the syntax of this extremely simple language.

Even then, the whole endeavor would have been out of reach of my brain I think.

You didn't start with "sudo mode: on" did you? That's what happens when you don't have the right privilege level.

Reply on news.ycombinator.com