Comment by ianbicking | Hacker News Reader

Comment by ianbicking | Hacker News Reader

Parent Full thread

I am frequently frustrated when I see people doing "studies" of LLMs where they don't put in the prompt engineering work. I came upon an example [1] recently where someone compared GPT-4 to Gemini Pro and Claude 2. The results hardly matter to me because they didn't put in the prompt work: they didn't give the model space to think (demanding it return only true/false), and they didn't give it higher-level instructions about the categorization, only vague instructions and a couple examples.

I think this often happens in order to be "objective" about the evaluation. I can see how it feels like cheating to coax the model to produce the answer you want. But... it's not! An off-handed prompt isn't more objective than a crafted prompt. You just haven't investigated its biases and flaws.

This lazy assessment is common everywhere, of course. It's one of the reasons bias gets into testing so easily: you setup a test and you assume that it is objective because you give everyone the same test with the same rubric. But if the subjects don't understand your terminology, or the proctor doesn't understand the subjects' terminology, it's easy to mistake misunderstanding for something else (intelligence, opinion, whatever you are testing for).

Systems based on communication need feedback loops, and that's just to get to the _starting point_. Prompt engineering is one of those feedback loops.

[1] https://www.vellum.ai/blog/best-at-text-classification-gemin...

> I can see how it feels like cheating to coax the model to produce the answer you want. But... it's not!

If it's for a single example, it is absolutely cheating. As an AI engineer this is a particular point of frustration where people complain because a large system can't return the result they want, when they were able to get the answer they wanted on their own with a lot of prompt hacking.

Each prompt is basically a point in latent space, and if you're "tweaking" the prompt what you're really doing is just re-rolling the dice until you land in a neighborhood closer the answer you want. You're not better at prompting, you just got lucky and are confusing that for insight.

Now if you're specific prompting trick works across a suite of evaluations, then you are probably on to something. But what people are doing in most cases is equivalent to performing some ritual before pulling the handle on a slot machine and then, when they finally win, claiming that they finally stumbled upon the correct ritual.

I understand your frustration, but it's more or less what is being advertised as possible and reasonable to try with current AIs. I don't think you'd see the articles as much (or at least not without more refutation) if it was more clear what to expect from AI in its current state. Companies are rushing to implement models into their products without considering all the qualifiers and methods to get good results that you mention in your post. I am not saying you're wrong, but it's hard for me to be frustrated with lay persons when the type of prompting you're frustrated about is exactly what they were told they can do.

AI is pretty fine in its current state for quick look-ups of stuff, but I absolutely agree with you -- without really focusing on the prompt given, the results will be suspect with current models. I am not meaning to discredit or disrespect AI, though I definitely do want to disrespect the way AI is being sold, neverminding how AI is portrayed in media.

> I am not meaning to discredit or disrespect AI

We are already at the point we need to watch our tone online. :)

I am curious how people see this evolving over time as the technology expands to more and more people. Do people get better at investing the time to craft the right prompts? Do shared custom GPTs etc become more the norm? Does the main AI become better at inferring our intent?

All of the above? It’s been shown that users interacting with the same agent (eg Siri) shift their language over time as they discover and internalize like works. LLMs both on their own and through future advancements will surely do the same. It seems natural to expect a symbiotic coevolution of both the prompter and the promptee’s languages.

Reply on news.ycombinator.com