Hacker News Reader

Comment by elicksaur | Hacker News Reader

“Prompting” is kind of a myth honestly.

Think about it, how much marginal influence does it really have if you say OP’s version vs a fully formed sentence? The keywords are what gets it in the area.

View on HN

That is not correct. The keywords mean nothing by themselves. To a transformer model, the relationships between words is where meaning resides. The model wants to answer your prompt with something that makes sense in context, so you have to help it out by providing that context. Feeding it a sentence fragment or a disjoint series of keywords may not have the desired effect.

To mix clichés, "I'm feeling lucky" isn't compatible with "Attention is all you need."

I find that providing more context and details initially leads to far more success for my uses. Once there’s a bit of context, I can start barking terms and commands tersely.

I find more hallucination - like when you're taught as a child to reflect back the question at the start of your answer.

If I am not careful, and "asking the question" in a way that assumes X, often X is assumed by the LLM to be true. ChatGPT has gotten better at correcting this with its web searches.

I am able to get better results with Claude when I ask for answers that include links to the relevant authoritative source of information. But sometimes it still makes up stuff that is not in the source material.

That’s fair. I think the difference here is that the entire context needed is provided.

If you’re having to explain an existing problem with edge cases, then sure, the context window needs the edge cases defined as well.

That’s the biggest problem I have on my local LLM use - limited context size compared to the big guys offerings.

Is this really the case, or is it the case with Claude etc because they've already been prompted to act as an "helpful assistant"? If you take a raw LLM and just type Google search style it might just continue it as a story or something.

Prompting is not a myth. The words of the prompt matter huge.

The problem with this prompt to me is not that it is not in a full sentence but that it isn't exact enough.

Probabilistically, "rust" is not about the programming language but the corrosion of metal. Then arrow.

Give the model basically nothing to work with then complain it doesn't do exactly what you want. Good luck with that.

Reply on news.ycombinator.com