Comment by ilaksh | Hacker News Reader

Comment by ilaksh | Hacker News Reader

This nonsense illustrates the most typical way that people misunderstand the word "engineering" in a software context. Software engineering and prompt engineering is not about the self-proclaimed level of rigor or formality that you apply. It's about the actual knowledge and processes used and especially their _effectiveness_ as measured in closed feedback loops.

But the starting point for this is that the term "prompt engineering" is an obvious exaggeration that people are using to promote a skill set which is real and very useful but a big stretch to describe as a whole new engineering discipline.

Regardless of what you call it, like software engineering, it really is a process of trial and error for the most part. With the capabilities of the latest OpenAI models, you should be aiming for a level of generality where most tasks are not going to have a simple answer that you can automatically check to create an accuracy score. EDIT: after thinking about it, there certainly are tasks that you could check for specific answers to create an accuracy score, but I still think it would make more sense in most cases to instead spend time iterating on user feedback rather than trying to think of comprehensive test cases on your own. There are a few things to know, such as the idea of providing examples, the necessary context, and telling the model to work step-by-step.

Actually I would say that there are two major things that could be improved in the engineering described in this article related to actually closing the feedback loops he mentions. He really should at least mention the possibility of coming up with a new prompt candidate after he was done with the first round of tests and also after the users found some problem cases.

The main thing is to close the feedback loops.

I believe it is a bit too much to call the article nonsense. The process described mirrors what we have been doing in Machine Learning for a long time: You setup a training set and validation set, put it through system under test and draw conclusions from the statistical analysis of the results.

writers are now word engineers, artists are image engineers, and politicians are now bullshit engineers.

Prompt engineering is just people putting effort into studying the behaviour of ML models and how input affects the output. They're more like ML psychologists than engineers. Calling themselves engineers just makes them feel better about being glorified prompt testers.

TIL that developing something like the Google Search ranking function is not engineering.

psychologists and sociologists use statistics to evaluate their results, does that make them engineers too?

Engineers of the human soul: https://en.wikipedia.org/wiki/Engineers_of_the_human_soul

so you're taking classification advice from joseph stalin? Maybe reconsider whether thats applicable to the situation or not.

That's beautiful, I'd never heard of it before. Thank you.

No, but it may mean that prompt engineering is a human science on the spectrum from phenomenology to scientism. ¯\_(ツ)_/¯

no it means prompt engineering is not engineering, thats just a euphemism used by those people to pretend to be doing something prestigious. They're doing engineering same way subway's sandwich assemblers do sandwich engineering - which is to say they aren't.

Right, like I said... on the spectrum... neither art nor beyond the apex of science as cult.

FWIW, Bon Appetit may not be clear on sandwich engineering, but it is clear on sandwich architecture:

https://www.bonappetit.com/columns/the-foodist/article/sandw...

I can certainly remember chefs in my city who met that standard.

I love “bullshit engineer” as a job description for politician.

100% agree (especially the politician part). Engineer became a status signum and people apply it to all kinds of stuff, diluting the meaning of the term. Engineering isn’t the right tool for the job in a lot of cases, and there’s no shame in that. It reminds me of “physics envy” in academia where other fields are applying methods and models that aren’t appropriate just because physics has high status.

OTOH, we already have social engineering and reverse engineering, which are now accepted terms, even though the engineering aspects are relatively weak. However, I must say I really still mind the term “data scientist”. Honestly it would be more appropriate with “alchemy”. It would also be quite hilarious. I wouldn’t mind being called a software alchemist.

I object. Please don’t take the word alchemy away from us amateur alchemists before we even figure out an economical way to transmute nuclear waste into gold.

Can we hold off on the cultural dilution until there’s an extant culture to dilute? :’(

I humbly accept this proposition. Do you have a suggestion on what term to use instead?

Perhaps "prompt doctor", "prompt physician', or "medical prompt professional" is a better term since they're making prompts better. People will probably like that much better since physicians seem to have high status.

It's actually significantly more appropriate as well considering the lack of understanding in cause and effect in both biology (physicians) and NNs (LLMs) compared to the better fundamental understanding from physics which influences design in engineering.

TLDR: "Prompt doctor" is the preferred term.

sound like more copium to me. These guys aren't doctors because they don't fix anything. They just study it to use it the best way to accomplish their goal. there's no engineering or doctoring involved. They're just looking for prestigious titles to co-opt for their ultra mundane completely replaceable job.

> These guys aren't doctors because they don't fix anything. They just study it to use it the best way to accomplish their goal.

By that logic, most doctors aren't doctors. The primary job performed by many medical professionals is simply that of convincing the patient to leave the office happier than when they came in, actual medical practice be damned. They "just study" the superficial aesthetics of medicine to "accomplish their goal" of making money and maintaining status (and of course, less cynically, making other people happy).

At the end of the day, these are all silly word games. Titles are meaningless in the search for truth. They exist only to flatter us as the infinitely fallible humans we like to be, and there isn't any point in picking fights over them.

I think prompt engineering is social engineering but for AI.

Of course, LLMs are like toddlers, prompt engineering is holding their hand so they get from A->B as best as possible. It's providing important context to frame output, guiding it along to "think" out it's processes, and breaking bigger problems down into smaller pieces. Essentially turning prompts into a series of smaller feedback loops - rather than dumping a big query and hoping the baseline toddler is smart enough to figure it out.

It's just about knowing the person you're interacting with extremely well, knowing their quirks and what motivates them, and carefully guiding them along, so they do what you want. Much like social engineering.

* Toddlers who've just ingested the AI equivalent of a bucket of sugar

Lmao, nobody's equating a prompt engineer with a civil or rail etc engineer.

Stop being pedantic "x engineer" has become a common phrase to describe someone who works on something related to the tech industry, the word engineer has expanded far beyond any original meanings.

The same pedantry could apply to the word "developer", too. It's really a weird sort of gatekeeping of a word.

The appropriate name for this is Prompt Designer, Model Tuners or Model Tuning imo.

Reply on news.ycombinator.com