Hacker News Reader

Comment by jiggawatts | Hacker News Reader

You can program GPT in English.

Let me repeat that: You can program GPT in English. ENGLISH!

You're complaining about the first nuclear test bomb being impractical and uninteresting. How will this change the world? That huge monstrosity had to be affixed to the top of a test gantry and took years of effort by a veritable army of the best and brightest to make! No way it could change war, or geopolitics, or anything. No way..

This is the day after Trinity. The bomb has gone off. A lot of physicists are very excited, some are terrified, and the military is salivating. The politicians are confused and scared, and the general public doesn't even know yet.

That doesn't mean the world hasn't changed, forever.

View on HN

> You can program GPT in English.

> Let me repeat that: You can program GPT in English. ENGLISH!

How?

Let me repeat that: How?

I had a little script that from time to time parses a list of jobs from a specific board, extracts some categories, inserts them into an SQLite and have a frontend that displays them to me in a way I want.

The board has since changed some things which would mean maybe 2 hours of commitment from me to update the script.

How do I program GPT in English. ENGLISH! To do that for me? What are the steps involved? I've been using ChatGPT and GPT-4 for awhile and I can't imagine what the steps are to make this happen without a lot of back and forth. I can't imagine how to program the infrastructure. I can't imagine how the API endpoint is more than a fancy autocomplete. I need help understanding what it means that I can program it in ENGLISH! (I can also program it in my country's language for what it's worth).

> That doesn't mean the world hasn't changed, forever.

I sort of agree with this.

Honestly don't think it will be long before gpt can read this comment, then politely ask you for the urls of the job board and your git repo and 2 seconds later you will have a pull request to review

You might find this interesting - https://github.com/Torantulino/Auto-GPT

> Auto-GPT is an experimental open-source application showcasing the capabilities of the GPT-4 language model. This program, driven by GPT-4, autonomously develops and manages businesses to increase net worth. As one of the first examples of GPT-4 running fully autonomously, Auto-GPT pushes the boundaries of what is possible with AI.

Perhaps, but I'm talking about right now. I would love to be able to do this as I have 1000 ideas and no time to try them out.

I have some scrapers built in Scrapy, and from my experimentation with GPT4, I bet you could paste in your scraper code, the html source from the website in question (at least the relevant part), and tell GPT4 to update your scraper and you'd get something that's at least 95% correct within 30 seconds.

> make this happen without a lot of back and forth

Perhaps this is the part you're missing. When I've watched people program with ChatGPT it _is_ a lot of back and forth because an enormous amount of context is able to be stored and back referenced. I.e. one wouldn't say "make me a Flappy Bird clone for iOS", they'd start with:

"Give me the code for a starter SpriteKit project". Then

"Now draw a sprite from bird.png and place it in the center of the screen".

"Now make it so the bird sprite will fall as if it's affected by gravity"

I won't bore anyone with how might one go from that all the way to a simple game, but I'm sure you see the idea. There are obviously _huge_ limitations to this approach and professionals will get hit them fast, but the proof is in the pudding: people who can barely code are producing real software through this approach. It's happening.

> Perhaps this is the part you're missing. When I've watched people program with ChatGPT it _is_ a lot of back and forth because an enormous amount of context is able to be stored and back referenced.

I've tried to build a lot of fun stuff with it so far. Haven't been able to properly 'program it in English' for anything non-trivial. Back and forth ended up in loops of not what I wanted. I'm just utterly confused at the difference in experiences I've had with it vs. what some people are preaching.

> There are obviously _huge_ limitations to this approach and professionals will get hit them fast, but the proof is in the pudding: people who can barely code are producing real software through this approach. It's happening.

I've had 4 product people I know try to create products using ChatGPT. All 4 of them basically got stuck on the first steps of whatever they were trying to do. "Where do I have to put this code?", "How do I put it online?", "How do I store user data?", "Where do I get a database from?". Basic questions to any professional, but to them it was impossible to overcome the obstacles from code to deployment.

I don't doubt that it's happening and it will become better in the future; I'm just having a hard time trying to grasp where some people are coming from when my experience as a professional, using it, has been mixed.

i've observed this schism between people who can get LLMs to produce useful output and people who are baffled, I think it's a mixture of two things:

expectations: using to the LLM to break problems into steps, suggest alternatives, using the LLM to help them think through the problem. I think this is the people using it to write emails - myself included, having a loop to dial in the letter allows me to write the letter without the activation energy needed to stare at a blank page

empathy: people who've spent enough time interacting with an LLM get to know how to boss it around. I think some people are able to put themselves in the LLMs shoes and imagine how to steer the attention into a particular semantic subspace where the model has enough context to say something useful.

GPT4 writes boilerplate python and javascript servers for me in one shot because I ask for precisely what I want and tell it what tools to use - I think because I have dialed in my expectation for what it's capable of and I learned how to ask in precise language, I get to be productive with GPT4's code output. Here's a transcript: https://poe.com/lookaroundyou/1512927999932108

Interesting point about empathy. Sorry I'm abusing the comment system to get back to your comment in the future.

The same way people used to write code in the early days: trial and error, and a boatload of cursing!

Let me give you a simple example. I had to deal with a desynced subtitle file recently. I described the exact nature of the desync (in terms like "at point X1 the offset is Y1, and at X2 it is Y2") to GPT-4 and asked it to write me a Python script to fix this. It did require a couple tweaks to run, but when it did, it "just worked".

"Automatic Language-Agnostic Subtitle Synchronization"

Link: https://github.com/kaegi/alass

It's basically magic.

You can't program GTP in anything if you can't program.

If your prompt is garbage then the output will be garbage and if you don't know how to program you won't even realize the output was garbage.

It's not the language part of programming language that is hard. It's the programming part because it means you have to have a good understanding of what you want. Just like a human programmer won't read your mind an AI programmer won't read your mind either.

But I can already foresee bosses dismissing employees that raise issues (performance, maintainability, scalability, etc., etc.) by saying "Look, the AI can do it. So if it can do it you can do it too.". I foresee this because I have already seen it.

> You can't program GTP in anything if you can't program.

That's why it makes this so interesting - this type of automation impacts our jobs directly. Of course, I'm not sure who would use this in a corporate codebase without legal concerns.

Work in a financial place, we have a contract with open AI so they don't hoover our user input. Normal URL is blocked.

> Let me repeat that: You can program GPT in English. ENGLISH!

The very existence of "prompt engineering", numerous discussions about how to prompt ChatGPT in order to get the result you want, etc. imply that while it may be in English, it still requires similar care and attention to do properly as a programming language does.

Which makes me wonder what the advantage of using English is. A formal language seems like it would be more productive and accurate.

For one, GPT-4 requires far less prompt engineering and generally interprets intent better.

The advantage of using English (natural language that is), the humans around you tend to speak it. I don't naturally speak powershell. Instead I want a script that searches for particular filenames, under a particular size, between a particular date in a directory path I specify. I told GPT I wanted that and in a few seconds it dumped out what I needed. It wrote the script in a formal language, which is then interpreted by the machine in an even more formal manner. Let the code deal with accuracy, and lets let language models argue back and forth with humans on intent.

> The advantage of using English (natural language that is), the humans around you tend to speak it.

This is true, but of limited utility. English is so bad at this sort of thing that even native-speaking humans are constantly misunderstanding each other. Especially when it comes to describing things and giving instructions.

That's why we have more formal languages (even ignoring programming languages) for when we need to speak with precision.

That's the other nice thing about ChatGPT - if you say it something and it misunderstands, you can correct it by saying, "no, actually, what I meant is ...". Which, again, is how people generally do that kind of thing outside of programming. The advantage is that you're still remaining on a much higher level of abstraction.

As far as formal languages... GPT doesn't know Lojban well, presumably because of its very small presence in the training data (and dearth of material in general). But it would be interesting to see how training on that specifically would turn out.

Every text interface will eventually include a half baked implementation of a programming language.

> Which, again, is how people generally do that kind of thing outside of programming.

Yes, and with people, that's insufficient if you really need confidence of understanding.

There's a reason that lawyers speak legalese, doctors speak medicalese, etc. These are highly structured languages to minimize confusion.

Even in less technical interactions, when you need to be sure that you understand what someone else is saying, you are taught to rephrase what they said and tell it back to them for confirmation. And there's still a large margin of error even then.

This is why, whenever I have an important conversation at work, I always send an email to the person telling them what I understood from our exchange. In part to check if I understood correctly, but also so that I have a record of the exchange to cover my ass if things go sideways because we didn't understand each other, but thought we did.

> You can program GPT in English. > Let me repeat that: You can program GPT in English. ENGLISH!

What does that mean to "program GPT"? Do you mean program (software) USING GPT?

I thought we already had COBOL, which is pretty much like English so business people can use it. Same for SQL.

And don't we already have lots of low-code or no-code tools? Why do we need to program with ChatGPT if we already are beyond programming?

Not the person you replied to, but I see it the same way. GPT is an English (and other natural language) compiler.

Not in the sense that you get a computer program out (though you can), but in the sense that it can automate anything without even needing a programming language, compiler, and domain specific UX.

Low code and no-code tools still require thinking like a programmer. You define what you need to do, then implement, then get results. GPT often lets you go directly from spec to results.

If the goal is programming, GPT is nothing special. If the goal is quickly reasoning over very abstract instructions, it’s amazing.

The trick is seeing the new use cases. It really does come back to the GUI revolution: if you want to list files in a directory, the CLI is just as good, maybe better. But GUI makes photoshop possible.

GPT makes it possible to say “summarize the status emails I sent over the past year, with one section per quarter and three bullet points per section”. And the magic is that is the programming.

> What does that mean to "program GPT"? Do you mean program (software) USING GPT?

A sibling comment already explained the second part of the question, but there is something I find more exciting. You can program GPT, as in you can tell it to change its behavior. The innumerable "jail break" prompts are just programs written in English which modify GPT itself. Like macros in lisp I guess. The first time I truly saw this potential was when someone showed me you don't actually have to change the temperature of chatGPT in code, you can just tell it to give low and high temperature answers in the prompt[1]. That's programming the model itself in english.

[1] https://news.ycombinator.com/item?id=34801574

The Ford Nucleon was a 1957 concept car that featured a compact nuclear reactor. Look at how well that prediction aged. It's apt that you mention the Trinity test, since 1950s inflated expectations of the applicability of nuclear everything are exactly where we are now.

Perhaps I could interest you in some Radium Water? It's new and trendy and good for your health.

Wikipedia has a fascinating article:

https://en.wikipedia.org/wiki/Radioactive_quackery

> Let me repeat that: You can program GPT in English. ENGLISH!

For problems that are fine being defined ambiguously. Try to program a database in English, let's see where it goes.

You laugh, but this is why SQL reads kinda-sorta like English. People have tried, and failed.

Meanwhile, if you give Chat GPT your database schema and ask it to write a SQL query for a report, it can do that for you.

If anyone wants to see the output of GPT4 when asked to define the tables and some sample queries for a hackernews clone in sqlite:

https://poe.com/lookaroundyou/1512927999932134

This is a very simple case that doesn’t reflect the complexity of a real project. Like so many attempts before to produce code, using a little effort, it degrades when the complexity level increases even slightly. Once there are more tables, ones that have names which cannot be easily translated from English, it breaks down quickly. These types of tools work ok for brand new projects, but work on existing projects will prove to be exponentially harder or more difficult than it is worth.

Nonetheless, it could prove useful for looking up algorithms, patterns, and generating boilerplate code. However, an important issue is will it generate similar code if queried at a later time? Not likely, which will make it less useful or result in an inconsistent codebase. Maybe you can request a version of the code generation? In-house code generators will generate consistent code, so it will be interesting to see how it is used in real projects.

Here's a more extreme example, using SQL as an API to give model access to game world state to reason about it.

https://gist.github.com/int19h/4f5b98bcb9fab124d308efc19e530...

Note that in this case it isn't even asked to write specific queries for specific tasks - it's just given one high-level task and the schema to work with.

You're right, though, that the effectiveness of this approach depends very much on schema design and things like descriptive table/column names etc (and even then sometimes you have to make it more explicit than a human would need). You really need to design the schema around the capabilities of the model for best results, which makes it that much harder to integrate with legacy stuff. Similarly, not all representations of data work equivalently well - originally, I gave the model direct access to the typed object graph, and it handles that much worse than SQL. So if your legacy software has a data model that is not easy to map to relational, too bad.

On the other hand, GPT-4 is already vastly better at this kind of task than GPT-3.5, so I think we can't assume that this will remain a limitation with larger models.

> You really need to design the schema around the capabilities of the model for best results, which makes it that much harder to integrate with legacy stuff.

This may end up being a feature of some high level frameworks … “compatible with ChatGPT” or “designed to work with xxx LLM”.

It will be very amusing if, eventually, our jobs as software engineers will be crafting bespoke AIs to maximize the efficiency of their use by an LLM.

Reply on news.ycombinator.com