AI Prompt Engineering Is Dead | Hacker News Reader

AI Prompt Engineering Is Dead

spectrum.ieee.org

70 pointsmfiguiere36 comments

Did anybody but the most desperate climbers ever honestly believe that "typing prompts into chatgpt" was ever going to be a high-paying full-time job?

Maybe. But man, is it blackpilling as an activity. Think about it: you're basically a sorcerer. You're trained in the arcane -- even to you -- art of conjuring spells. Nobody, even the savviest practicioners, know or can know how the words produce the magical effects. They just know the words that (probably) do.

It's medieval anti-enlightenment black magic. The genies are black box tangled confections of brute force computing power in reach of only nation states or megacorps, which also don't know how they work but try to weave them along their agendas.

Programming was supposed to be a conquest of reason over the darkness of ignorance, and this is the exact opposite. :D We're back to wizard whispering to spirits. "Prompt engineer" a circle of salt and a burnt white rabbit to invoke Eschmoûn for a good harvest.

What a depressing thing to aspire toward.

This can be said for anything.

An electrician is a magician already. A painter is a sorcerer. A programmer creates life.

I think, in the limit, you're right.

I think it's cool I can understand an algorithm and what it does, but, once it becomes machine code, my understanding utterly ends. What's the CPU doing with it? Have I even delidded my CPU to make sure there's actually silicon in there and not pixie dust? Forget the computer; how does my own brain work? Nobody even knows how brains store memories. A neurologist might know a lot more than I do, but maybe this is all a solipsistic simulation where I'm a brain in a jar and the neurologist is just an NPC controlled by a fancy AI.

Perhaps, in reality, the feeling that you know what's going on was always a foolish conceit. It's magic all the way down. Astronaut with gun: always has been.

I guess I have to find comfort in degrees because otherwise I feel very helpless and stupefied by that way to view the world.

A more relevant comparison would be the people on youtube demonstrating how to "ground" your bed by sleeping on a mesh wired to a pole in the ground.

I had a discussion on HN with someone extolling the virtues of ChatGPT extracting data from PDF's. It took me forever to get them to start acknowledging the infamous 3rd step (correct prompting). And apparently I'm just unskilled for saying it wasn't magic but required correct input.

Not that ChatGPT can't do it, just that a lot of people seem to think it's going to be like talking to the star trek computer.

Probably not the point you were making but OpenAI is paying (a small group of) people pretty high hourly rates (like 70-100$/hr) through their “redteam” program

This is vastly different from other, non-AI companies hiring people to prompt AIs for them. And a quick google shows that it's not a full-time job at all.

This sounds more like QA testing. Preventing situations that could hurt the public image of the company, or similar.

Anything can be described dismissively. I do think prompt engineering was overhyped but I don't think that it's that obvious or that something of that nature couldn't have been a valuable skill in the future.

So many desk jobs that the professional managerial class does today can probably be described similarly. That doesn't mean that they can't still be high-paying.

Unfortunately yes, I think there are many that thought precisely that

Well, since being a manager is basically 'telling other people what to do in a way such that they'll actually do it' — yes. /s

"Just develop a scoring metric so that the system itself can tell whether one prompt is better than another, and then just let the model optimize itself."

The word "just" is doing a LOT of work there!

[ and it's already too late for this, but before posting snarky notes about prompt engineering as a profession you should know that this article isn't about that - it's about tools like DSPy that automatically optimize prompts ]

Prompt engineering is just a fancy way of saying "rephrase the question until you get the answer you're looking for." People do that all the time when talking to each other. I don't see how better AI would change that.

It's the same story with search engines. Google et al may like to say they've tuned the engine to be "optimal" at guessing what people "really want." In practice, I've found the opposite to be the case. The damn thing interprets my query in such a liberal way that it returns a bunch of irrelevant garbage. Maybe their numbers show that this works better for the average person. But with people who really know what they're looking for (like me) it seems to have gotten far worse.

I had a co-worker who used to call it "Google Fu" because I would search for something but he had a knack for always getting better search results.

The title is click baiting.

Prompt engineering is still important.

From a philosophical point, it always makes a difference how you phrase a problem. If you formalize a problem in Lean code, it's easy to understand for a proof assistant but not for most humans (even mathematicians or programmers).

So rephrasing the problem in human language makes it easier to grasp and comprehend. And given the dataset the AI is trained on, there might be certain style it prefers. If you train it on Lean code, better input lean code. Also, more context makes it usually easier to solve the problem.

It’s important the same way code organization is important.

You need to do a good job at it, but it’s not something anyone is going to get a full time job to do.

To expand on your excellent point: the danger to programmer jobs is that one day product owners might be able to cut out the middleman and get the job done without the programmers. On the other hand, being able to communicate clearly with the LLM and knowing its quirks becomes even more important. And for quite a while one wouldn't actually trust the output of a model.

> the danger to programmer jobs is that one day product owners might be able to cut out the middleman and get the job done without the programmers

To do that, they'd need to have some kind of coherent detailed plans that actually made sense in the context of all the minutia of the rest of the system. If the product were built of fingerpaint and good intentions, POs would still need programmers.

If you have been hired as a "Prompt Engineer", can you please explain your work? Are you just finding better ways to communicate clearly in human language to the current model you are interacting with? Or are you doing statistical analyses of prompts, etc? It is hard for me not think a Prompt Engineer as most basic is someone comfortable and knowledgeable about how to express a concept in human language to an LLM, which to me, really sounds like at minimum: clearly communicate with little ambiguity.

I'm not a full time "prompt engineer" but it's a small role in the work that I do for my business.

Prompt engineering is kinda like a basic form of data science. You have a dataset and some manually labelled results, and you hypothesize on prompts and try to improve on some metric. You'd be surprised how the tiniest of changes will alter a result. Thinks like a hyphen, semicolon, or capitalization can sway the metric. It's very finicky and very annoying.

Tokenization errors are also really, really annoying to deal with.

e.g. why does the JSON output have silly whitespace/quotation sometimes? Obviously it's because the first token the model output was `{` and not `{"` like it should be. Obviously.

Totally. When using GPT I exclusively use function calling, which is far more reliable for JSON output. When using other models I don't even bother with strict JSON because of the error rate - and instead opt for HTML which is more forgiving when parsing.

nono, the same thing happens with OpenAI's JSON mode and function calling. It does output parsable JSON very reliably but it comes out mangled with a bunch of whitespace sometimes.

GPT-4-turbo's output context window is limited to 4096 so fixing this is relevant. You can use logit_bias for it.

It's not as simple as being descriptive, but what you're of descriptive and knowing the terms to use.

You also need to know the tools to use and there's a lot of settings you play with that are hidden from the user in mainstream apps.

> "Prompt engineering is best done by the model"

Well, That would be ideal, but if I type in "Middle-aged white male in full plate armor standing on a battlefield resting on a full tower shield" I likely will want to further modify the result, or style, or detail level. There almost certainly will continue to be "hacks" to get it stylized as desired. Even if I say "Painting of..." there's still a huge range of options.

I understand and agree that it's desirable to get AI prompts as close to natural language, but how do you quantify a level of stylization in natural language? "A very very very very Michelangelo style painting of a slightly slightly slightly Middle-aged white male..."

I think prompt engineering will change quickly, and to keep up, it could potentially be a 'profession' that is very specific to the model. I don't think that's a bad thing, but I would think/agree that it will likely not employ many people at all.

"Prompt engineering" is just specification writing but with a LLM instead of a person as the one implementing the spec's requirements. Natural language is imprecise, precisely specifying requirements is hard, and LLMs can't read your mind any more than other humans can.

Maybe it would help if you actually understood the specific qualities of "very very Michelangelo style" and could describe those, instead?

Prompt engineer was likely never a career path, I’d imagine most software devs knew this from the jump. I think any engineer can work with the modern LLM solutions available — considering software engineering was already a form of pseudo-prompt engineering, but the feedback loop was with yourself, google, and stack overflow.

The product I’m building is predicated on prompt engineering and it works really well for us — our results need to be as accurate and structured as possible so they can be re-used as software dependencies. We do this by auto-generating a context on the fly and tailoring it to your specific data sources (databases, APIs), sampling data, and so on. We do both manual and "auto" tuning of the context to provide the best results.

I'm interested in hearing if anyone has used DSPy (https://github.com/stanfordnlp/dspy) just for prompt optimization for GPT-3.5 or GPT-4. Was it worth the effort and much better than manual prompt iteration? Was the optimized prompt some weird incantation? Any other insights?

I can’t believe any researcher that measures prompt strength based on how well an LLM does math.

"AI Prompt Engineering" is just learning how to use a model, and it will be a thing until AIs learn to literally learn our mind.

Even if the system adds a layer with "autotuned prompts" as the article calls them, some prompts will inevitably work better than others, and some models will have to be "spoken to" in a certain way or use certain keywords to make it work in a specific way, but that knowledge may not work with all models.

For example, recently I've discovered that adding "no yappin" at the end of the prompt makes ChatGPT 4 go directly to the interesting part, but if I use it on another model it could do nothing, or maybe it gets confused and starts spouting garbage, or it may get pissy and scold me for being rude.

This might come off as a joke, but I don't doubt at all that it will soon be reality, as different models will have different "personalities". This term is not to be taken literally, but it gets close enough to the idea of representing the internal workings of a LLM.

Little tricks and hacks, like going as far as bribing or threatening the LLM to get a better or specific kind of output, mostly model-specific.

The future will be interesting for sure.

Eventually, as these models get better at language (and thus converge and become more similar to each other) prompts will converge as well. (more so than they already have)

To adequately use a large language model it is necessary to have your own large language. Perhaps a prompt engineer would be more appropriately termed a “Cunning linguist”

I don’t think it ever was alive

My ongoing hypothesis [1] is that natural language has intelligence (or to say it another way, natural language has embedded structure that harmonizes with the structure of problems that we face[2]).

If correct, there are a few relevant conclusions that you can draw. The first is that prompt engineering is a real thing. By talking "correctly" to the LLM you're providing it with additional information that will allow it to give you more legitimate answers. This might help explain these threads where I'm seeing a lot of comments from people who rave non-stop about all of the work they're able to get LLMs do for them AND also people who cannot get LLMs to do anything worth doing (I'm included in this latter group). The former know how to "talk right" to the LLM.

While I think prompt engineering is real, I don't think it's necessarily important. "Talking correctly" is nebulous and resists our ability to categorize. The metric is "gets good answers from LLM", but in order to know if the answers are good you have to also know the reality. It's pretty obvious (at least to me) that if you have to choose between speaking in a useful way and knowing the reality that you would just choose knowing reality[3].

The other thing is that this means that LLMs are a dead-ish end. Because they're approximating the intelligence in natural language, they'll never be smarter than our collective words. Sure, we'll be able to make cheaper, smaller, and more specialized LLMs, but they'll need another non-LLM component to reach the next level of usefulness.

[1] - And I mostly keep mentioning it to see how people react. I've yet to see a "this is definitely wrong because ..." type of response. I'm hoping to see one that actually has some meat to it.

[2] - So, the idea is that word2vec allows you to do vector math of the pattern: KING - QUEEN = v; Man - V = Woman. Great, so our grammar has some sort of algebraic structures inside of it. And this sort of makes sense that this would happen. Kind of an information theory version of frequent messages should be small. Only here, it's natural language should have a structure that mirrors the structure of problems we care about. People who talk the right way have a tendency to solve problems that matter and natural language evolves in the direction that enables people to be successful.

The problem then is that some problems are simply too complex to embed their structure into natural language grammar. Like rocket science, brain surgery, or OS construction. The other problem is that some problems are simply too niche to develop a jargon. And finally, some problems have a structure that we haven't seen before. In any of these cases, I expect LLMs to fail to be useful.

[3] - I suppose there could exist some world in which the knowers are able to certify the talkers and then maybe that's easier than teaching unknowers to be knowers. But it feels like it's a bit risky. If the talkers ever get off base, then it's not like anyone is going to notice until it's too late.

Reply on news.ycombinator.com