Hacker News Reader

Comment by magicalhippo | Hacker News Reader

Gemini seems to be fairly good at keeping the custom instructions in mind. In mine I've told it to not assume my ideas are good and provide critique where appropriate. And I find it does that fairly well.

View on HN

Same. This works fine for Claude in my experience. My user prompt is fairly large and encourages certain behaviours I want to see, which involves being critical and considering the strengths and weaknesses of ideas before drawing conclusions. As someone else mentioned, there does seem to be a phenomenon where saying DO NOT DO X causes a sort of attention bias on X which can lead to X occurring despite the clear instructions. I've never empirically tested that, I've just noticed better results over the years when telling it what paths to stick to rather than specific things not do to.

That happens with humans too :) It's why positive feedback that draws attention to the behavior you want to encourage often works better. "Attention" is lower level and more fundamental than reasoning by syllogism.

> there does seem to be a phenomenon where saying DO NOT DO X causes a sort of attention bias on X which can lead to X occurring despite the clear instructions

It's a thing with people too[1], ie do not think about a white bear.

[1]: https://en.wikipedia.org/wiki/Ironic_process_theory

Don't think about what?

I will admit that I was very pleasantly surprised by gemini lately. I was away from my PC and tried it on a whim for a semi-random consumer question that led into smaller rabbit hole. It seemed helpful enough and focused on what I tried to get while still pushing back when my 'solutions' seemed out of whack.

> Gemini seems to be fairly good at keeping the custom instructions in mind.

Unless those instructions are "stop providing links to you for every question ".

I think that's part of the RAG pipeline, at least to me it looks to be separate from the model output. Models are notorious for getting URLs wrong so makes sense to me to handle it separately.

Reply on news.ycombinator.com