Comment by ThePhysicist | Hacker News Reader

Comment by ThePhysicist | Hacker News Reader

Parent Full thread

For prototyping Chat-GPT (4.0) is a big deal. I had it write PoCs of pretty complex systems combining different state-of-the art technologies, and it mastered all of it brilliantly. What amazes me most is how simple and elegant its solutions are. When you ask something (e.g. build a horizontally scalable data store with a Raft-based consensus mechanism in Rust) it produces code that does just that and nothing else. Lots of programmers, myself included, would struggle with this and include at least some extraneous complexity, but Chat-GPT goes right for what was asked and seems to find the most succint way to achieve it.

I haven't felt so excited about programming in a long time, because I can now build PoCs that would take me days or weeks to do in an hour or even less. That's a real game changer, and it helps me overcome this internal friction of "I know this is possible to build but it will take me a long time to figure out an actual approach".

How is it ChatGPT 4 is like a few days old and people are reviewing its ability to "build complex systems with state of the art technologies" like they've been using it for years?

Reminds me of "Seeking developer with 10 years of experience in <brand new> framework"

> How is it ChatGPT 4 is like a few days old and people are reviewing its ability to "build complex systems with state of the art technologies" like they've been using it for years?

Why... why can't we reviewing its ability to "build complex system" because it's a few days old...?

I'm pretty sure here "complex" is relative to what ChatGPT can do before, not relative to, say, what NASA did before.

> Reminds me of "Seeking developer with 10 years of experience in <brand new> framework"

AI is not human. The anthropomorphism is quite crazy here.

>AI is not human. The anthropomorphism is quite crazy here.

You misunderstand. My point isn't that ChatGPT is the developer here, it's that ChatGPT is the "framework".

I spent half an hour today trying to get ChatGPT to write a simple anchor tag in HTML and didn't get a single correct response. I'd love to see a screencast video from one of you people claiming that ChatGPT is actually returning huge amounts of useful code that is ready to use. I'm incredibly skeptical!

Input: Give an example of anchor tags in html

Output: Sure, here is an example of how to create an anchor tag in HTML:

<a href="https://www.example.com">Click me!</a>

In this example, the text "Click me!" will be displayed as a hyperlink. When a user clicks on the hyperlink, they will be redirected to the website specified in the href attribute, which in this case is https://www.example.com.

I would love to see a screencast video from you of what you asked ChatGPT and what it responded with, as I find it hard to believe that it couldn't write an anchor tag.

Could you provide an example of this? What kind of prompts are you using, and what does a complex system of different state-of-the art technology would look like?

Sincere question as I don't get how to use it for things larger than simple scripts.

You just talk to it like you would talk to another engineer. I e.g. first describe what I want to achieve and ask it to give me an architecture. I then e.g. ask to break down the work into different smaller tasks, and ask it to do each task, adding to what it wrote before. A bit hard to reproduce all prompts here and the UI doesn't allow to share, but it comes quite natural I find. Sometimes you need to ask it to continue as the output just stops, but other than that it's like talking to a colleague (with superhuman powers).

I usually ask it to describe the concept first. To make sure it is in the right space. Then I say something like "please write a c++ version of that".

It will do a decent job at it.

Last night a friend who refuses to get an account to use it was having an issue with a cisco router. I put in what he was trying to do. It was not right and gave him an error. I fed that error back in and it realized he had a different version and gave me a better way to do exactly what he wanted. It had kept the context and said 'oh some routers do not have that command here is another way to do it'. He had spent weeks googling around for the answer. I had it in under a half hour and I had never used the interface he was changing before.

Then you can turn around and ask it to write a horror story about a monster that devours couches. It will make something up (it is very good at that). Then you can say 'oh put that in the style of the SCP foundation'. It will.

This tool is wildly interesting. I think dismissing it is a bad idea. I look forward to using this thing as it gets better.

I have been trying this, but it crashes every single time. I end up getting a huge amount of output, but I’m guessing that it exceeds the time limit and crashes losing the entire output.

Maybe I’m using it wrong, but I just describe the system that I want to build and it starts listing out multiple files. Usually by the 3rd or 4th file, it crashes with an error.

I’m not sure how to reduce the output so it doesn’t crash! I have ChatGPT Plus using GPT4, fwiw.

Did you have this problem too, and do you remedy by giving it smaller bits of information per question?

Just write "You were interrupted, please continue." - that will prompt it to pick up its train of thoughts and continue. Strange times we live in...

It seems a lot of prompts are polite (e.g. “please”). Weird, since I don’t think we ask Google politely?

How does being polite in your prompts help you?

It makes a big difference with Bing bot.

Try a prompt where you want it to create a list of something in two versions:

* Create a list of the top 20 blah sorted by blah

It will typically return just a few and then refuse to give you more. Then try:

* Acting as a conscientious and resourceful research assistant, use your knowledge and initiative to create a list of the top 20 blah sorted by blah.

You get much better results.

Lately I find that you can reduce its hallucinations somewhat by adding something like:

* If you are unsure of any information put in "??". Do not fabricate information. I understand that not all information is available and appreciate your work.

This is all a bit much - having to coddle an AI, but it's possible to understand why it would be the case: it's trained on lots of human interactions and apparently responds better to a friendly interaction with keywords that set it up to do a good job.

The South Park: Post Covid special had Amazon Alexa androids who were girlfriends to some characters. The boyfriend had to walk on eggshells with some topics or Alexa would go into a screaming fit.

Kind of prescient.

https://m.youtube.com/watch?v=lugeruSbnAE

Out of curiosity, how do you know that it’s correct?

Test and reiterate. I don't think you should take whatever is coming out of chatGPT verbatim. It should be used as an interactive tool to amplify your knowledge.

GPT-4 for coding is very powerful but can be dangerous. I asked it to write a data retention script that would save X number of backups depending on the number of days/weeks/month/years from the present day. It's first attempt would have worked the first day, but running the script consecutive days would eventually delete all backups. Once I told GPT-4 to take into account the script would be run each day, it "apologized" and produced a script that worked.

Testing does not show correctness, it can only show incorrectness.

Similar to how you know someone's stackoverflow answer is correct. Read the code, find the documentation of API calls, think of any missed cases, write tests etc.

So basically the same as writing it yourself: just skipping the writing part.

So should we be taking about GPT as an occasional text editor replacement? I honestly think that's more accurate take than most of the ones I have seen.

Have you seen the GPT4 demo? A few interesting things that make it seem much more than an occasional text editor replacement. One was having it write an entire program. It needed a few prompts to fix some bugs it had, but it worked. The program was then used for the rest of the demo. The program even had an obscure bug that it solved by having the demo-er paste the error + the entire docs page. The second demo was having it write a html page based on a picture drawing on a napkin of a website, and then running the html.

It's honestly pretty impressive. I already use chatgpt a lot (doing a lot of google sheets scripting/formula stuff and the docs + syntax are horrendous) and chatgpt helps me find the correct syntax much faster than I'd otherwise be able to

There is a huge difference between writing a program, and writing the program you want right now.

It's tricky to conceptualize because the entire narrative we are familiar with has personified GPT. It may be impressive, but GPT is completely different from humans.

If a human is capable of writing a program, that is because they understand conceptually how and why. GPT doesn't. GPT's ability to write a program is entirely dependent on the content of its training corpus: no how, no why, only what.

I'll put it another way: If a human is not able to write a program, it is because they don't understand conceptually how. If GPT is not able to write a program, it is because either the desired parts, or the necessary pattern that puts them together, does not exist in the training corpus; or because the prompt didn't yield a continuation that followed that arbitrary pattern.

The results are impressive because humans are impressive. GPT doesn't interact with the domain of all possible written text. It deals with the domain of all possible patterns of tokens from the text it was given. It's only given text that was intentionally written by humans. It exists in a world of signal: no noise. It can still only guess, but the guess will always be constructed out of the category of text humans choose to write.

Inference models are a completely different approach to language, and confusing them with human behavior is an easy way to make impossible predictions about what they can and cannot accomplish.

It's a rubber duck that talks back. Massive amplifier on software productivity.

Reading the code, and running it. You can ask it for a complete project tree and even test scripts (you have to ask it to continue as it gets interrupted sometimes). I e.g. had it write a simple database engine with an API, and it produced Rust code as well as curl-based examples to test the database. It even correctly added up the data items it sent into the DB via curl to show what the output would probably look like. Pure sci-fi technology.

The author addressed that rather at length in the article. Don’t. Trust. Code.

That it originated from ChatGPT instead of any number of other sources doesn’t change that.

Same way we know that any piece of code is correct.

Debugging, testing.

Reply on news.ycombinator.com