A guide to prompting AI, for what it is worth
oneusefulthing.org
oneusefulthing.org
> Being “good at prompting” is a temporary state of affairs.
Valid point. Sam A said he thinks prompt engineering won't even be a thing in 5 years. And in the shorter term, prompts that work with a specific model like GPT-4 might not work well for future models, or even updates to the same model.
That said, I see prompt engineering as the beginning of a new paradigm of "intent engineering"--where developers use AI to understand and anticipate user intent with minimal user effort. It'll be fun to see what that looks like in 5 years.
[0] https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg
I think part of this comes from the fact that the same input will produce a different output.
No that SQL has no value, but this is exactly what ORMs do for a lot of people. They stay in their language instead of having to learn SQL (not saying this is ideal).
But with ChatGPT, even if you give it a terrible prompt, it can still "get" what you're trying to say. You can keep chatting until you get the image you want. It's way easier than with Txt2img, which can be pretty unforgiving.
And those courses? Total scam, don't bother with them.
Who cares what Sam A thinks on the matter. Of course he is going to say things like that he wants to make his product sound as amazing as possible.
I don’t understand what this could mean. I have to do “prompt engineering” when I talk to other people all the time - when I need to ask them to do tasks, when I need to clarify requirements, etc. As long as we’re communicating through text and not mind reading, some level of “engineering” will be required. AI is intelligent but it’s not magic.
---
It has zero effect. Your prompt consists of the OpenAI prompt + your conversation so far. There is no cross-prompt learning, knowledge or intelligence [except for, in the future, what OpenAI decides to include in model training of new releases]. The main reason for this is there are a limited number of tokens that can be used as the prompt. If you "include everything we learned in this chat session" it would have to include the entire chat session as part of all future prompts and you would quickly run out of tokens for the prompt. Training happens on a corpus, but not during regular use. Perceived "learning" is just context provided by the session-limited prompt.
Write a followup email to A about B with X,Y,Z characteristics.
[iterative chat]
Give me back a prompt for next time.
Ok: “Write a followup email to A about B with W,X,Y characteristics.”
I thought that was understood. I basically ask it to improve my initial prompt, like asking it how could I have asked this better. Sometimes it gets good results, like when it taught me how to pose formatting to it. I did not know how to make it output in a certain user defined format, it never occured to me that I can just give it an example, the modified prompt taught me that trick. Now it's super common everywhere and I think is the basis of a lot of langchain.
If this actually happens, I would imagine that talking to an LLM would be a combination of formal syntax and natural language, so the task will look more like software engineering than black magic.
This reminds me of Inform 7 (where you can use "natural language" to program text adventure games) and "visual" programming languages like Pure Data.
They are fine for simple things, but when you want something complex or want to debug something they can become a nightmare.
LLMs are currently better in many ways than Inform 7, and they'll likely get better still, but there will likely still be a role for formal languages. Fortunately, LLMs can take formal language as input as well, and with the addition of plugins, they'll be able to execute programs written in formal languages.
> Please function as a TTRPG engine.
> All characters start with 100 tokens representing LIFE, which can range from 0 to 100.
> Health scales with LIFE. At 0 LIFE, you are dead. At 100 LIFE, you are in peak physical condition.
Etc.
The problem (at least, the problem I had) with Inform 7 is that it's not really natural language; it still has a strict formal grammar defining its syntax, it's just that the grammar has lots of "synonyms"; multiple syntaxes for the same thing. But it's still following strict rules, and those rules can't cover everything. This leads to situations where you write some "plain English" and it seemingly understands it, but then you write tiny variation and it throws up a syntax error.
Inform 7 is much more of a DSL based on principles derived from natural language than natural language itself.
Watching whole communities crowd-source/stumble their way into basic reference question principles is kind of fun, though.
This is really the essence of why most software development is hard. Aside from a few problems whose solutions can be mathematically or logically defined, programmers are working on software to do things humans want to do to affect the external world in some may. They're what Meir M. Lehman calls E-programs: programs to model human and social activities. The program becomes part of the world it models, e.g. air traffic control.
Acknowledging the inability of humans to understand what they want shows the folly of trying to get all the requirements "up front" before starting the coding.
Pulling from my hardware background...
When you design with FPGA's and write code using HDL (Hardware Description Language; Verilog, VHDL) you learn to write code the compile will use to infer the hardware structures you are after. Despite what it may seem, you are not programming, you are describing hardware and, in some cases, use use idioms or structures that will result in the desired outcome.
In some ways prompt engineering can be thought of as learning how to cause the AI to deliver the desired result using the most efficient prompt text.
You could foresee that under the covers there is always going to be some prompting but it's going to be performed rarely by few people?
The most important part is that the prompt has the necessary information so that it can be interpreted correctly. The other important part is, if you're trying to get a response you can feed to a computer (e.g. raw JSON), you need to really clearly specify this. And even then the current LLMs are really bad at stopping or providing invalid output; so bad, we may need to create another type of language model which takes LLM "English" output and converts it into raw data.
LLMs are (or at least GPT-4 is) also really good at selective attention. Even if you slip in a subtle detail the model is good at picking it up.
That's my impression. I just write what I want chatGPT to do, and eventually refine according to the answers. Why has "engineering" been added to "prompt"? Why not simply "prompting writing"? Or "Better prompt writing"? (is it not simply equivalent to "how to write better")?
We could declare certain pieces of the prompt as "variables," and certain defined operations that have proven deterministic output as "functions." All so that what we want the computer to do can be rigorously defined and tested.
There might be a tiny ghost in an LLM machine, but it's very unlikely that it has the degree of self-awareness required to assess whether or not a prompt was, in its own experience, unambiguous. It's a reflection of its training data, and it has no or little sense of self.
On a technical note, there isn't abstract comprehension going on, like the discovery that chess dosen't require intelligence, what happens when creative people recognize they don't either ;p