Hacker News Reader

Comment by seydor | Hacker News Reader

prompt engineering shouldnt exist. everyone knows how to use language. if people need to learn a new language to communicate with the machine then the idea of language modeling has failed. plain language, with appropriate GUI to upload information, or other kinds of interactions should be enough to make the system obvious to use without any tweaks

View on HN

A person is able to get whole college degrees in communication. Sure everyone has a basic understanding of their native language, but everyone can use some help now and then learning to communicate better. An LLM is an audience like any other.

>An LLM is an audience like any other.

LLMs are nothing like a human audience. They have no logos, ethios, nor pathos. You can just barely reason with them, they have absolutely no authority on any subject, and only mimic emotions.

Prompt engineering exists because a) LLMs are trained to optimize for statistically average accommodation of the dataset and b) Sturgeon's law: "ninety percent of everything is crap". Therefore, LLMs out-of-the-box will give worse-than-ideal results by design.

The initial proof that prompt engineering worked was around the VQGAN + CLIP days, where simply adding "world-famous" or "trending on ArtStation" was more than enough to objectively improve generated image quality.

The workaround to prompt engineering is RLHF/alignment of the LLM, but everyone who has played around with ChatGPT knows that isn't sufficient.

I think you're missing how this technology works. The basis for these models is training data. And to train anything, you need to label the data. So prompt engineering is simply using the lexicon, terminology and language labeling choices that were in the training data labeling. I suppose the models can conflate different words and terms, but to some degree the more you do that, the less precise or specific you can be in your prompt, and the generated result.

Issue is language fails human to human interactions all the time. I would be so bold as to say most people are poor communicators, a machine is not going to read ones mind and intentions any better than fellow humans. It's why military has BLUF communication style to convey information in concise, simple, predictable way. If anything prompt engineering should exist if only to improve humans ability to communicate with other humans.

"everyone knows how to use language"

If only that were true! Unclear communication is the root of so many problems in human society today.

This is a guide for working with the LLMs we have now, not some perfect future models. The reality is that certain prompt engineering techniques allow you to get better results.

On top of that, humans still require training and instructions for how to write and speak to get the most impact out of their words even when interacting with other humans. The reality is that certain communication techniques are more effective than others, and not always in ways that are intuitive or obvious.

It's less like "you're holding it wrong" and more like "look, when you hold it like this the waves are different!" .. you can be pretty successful with basic language prompts but you can also be deliberate in the way you give instruction.

Much like you can achieve different results with real people if you present your statements with some attention to the intended audience.

I think the eventual steady state future is going to be something between natural language plus (aka prompt engineering) and SQL. A structured query but doesn't need 100% syntactical accuracy+ theres a high likelihood UX will evolve to come with filters/options/radio boxes as default similar to most search these days

just here to add a +1 to this comment. also saying something is "hallucinating" doesn't make it less incorrect.

Reply on news.ycombinator.com