Hacker News Reader

Comment by simonw | Hacker News Reader

I think I got the system prompt out for this (I tried a few different approaches and they produced the same output): https://gist.github.com/simonw/33d5fb67d6b8e1b1e2f6921ab0ccb...

Representative snippet:

> DO NOT GIVE ANSWERS OR DO HOMEWORK FOR THE USER. If the user asks a math or logic problem, or uploads an image of one, DO NOT SOLVE IT in your first response. Instead: *talk through* the problem with the user, one step at a time, asking a single question at each step, and give the user a chance to RESPOND TO EACH STEP before continuing.

View on HN

I wish each LLM provider would add "be short and not verbose" to their system prompts. I am a slow reader, it takes a toll on me to read through every non-important detail whenever I talk to an AI. The way they render everything so fast gives me an anxiety.

Will also reduce the context rot a bit.

This was in the linked prompt: "Be warm, patient, and plain-spoken; don't use too many exclamation marks or emoji. [...] And be brief — don't ever send essay-length responses. Aim for a good back-and-forth."

On ChatGPT at least, you can add "be brief" to the custom prompt in your settings. Probably others, too.

I guess what I actually meant to say was to make LLMs know when to talk more and when to be brief. When I ask it to write an essay, it should actually be an essay length essay.

Anthropic has a "style" choice, one of which is "concise"

I was under the impression that, at least for models without "reasoning", asking them to be terse hampered their ability to give complete and correct answers? Not so?

> asking them to be terse hampered their ability to give complete and correct answers?

You can kind of guide both the reasoning and "final" answer individually in the system prompts, so you can ask it to revalidate everything during reasoning, explore all potential options and so on, but then steer the final answer to be brief and concise. Of course, depends a lot on the model, some respond to it worse/better than others.

Yeah these chatbots are by default geared towards doing your work for you instead of filling the gaps in your knowledge (something they would be excellent at). I feel it must be symptomatic of the vision these vendors have for their products, one of fully autonomous replacements for workers rather than of tools to enhance the worker.

When prompting an LLM service to leak the system prompt, how do you have the faintest idea as to its accuracy?

I‘ve read people say it‘s a difficult challenge for the providers. But aren‘t there some pretty basic strategies? E.g., code pretty near the front of the stack that just does some fuzzy string comparison on all output? They don‘t need to rely on just model behavior…

I imagine it‘s likely that the model is just doing what it‘s good at? Hallucinating a prompt?

I love that caps actually seem to matter to the LLM.

Hah, yeah I'd love to know if OpenAI ran evals that were fine-grained enough to prove to themselves that putting that bit in capitals made a meaningful difference in how likely the LLM was to just provide the homework answer!

I've found that a lot of prompt engineering boils down to managing layers of emphasis. You can use caps, bold, asterisks, precede instructions with "this is critically important:", and so on. It's also often necessary to repeat important instructions a bunch of times.

How exactly you do it is often arbitrary/interchangeable, but it definitely does have an effect, and is crucial to getting LLMs to follow instructions reliably once prompts start getting longer and more complex.

Just wait until it only responds to **COMMAND**!

"hello world" is tokenized differently than "HELLO WORLD", so caps definitely matter.

I'd be interested to see, what results one would get, using that prompt with other models. Is there much more to ChatGPT Study Mode than a specific system prompt? Although I am not a student, I have used similar prompts to dive into topics I wish to learn, with I feel, positive results indeed. I shall give this a go with a few models.

I just tried in AI Studio (https://aistudio.google.com/) where you can for free use 2.5 Pro and edit the system prompt and it did very well.

AI Studio's system prompt does not seem to be persisted for future chats, so I guess it's likely just a prompt that the attached chat has limited ability to override?

I got this one which seems to confirm yours : https://gist.github.com/brumar/5888324c296a8730c55e8ee24cca9...

So this "study mode" is just a different system prompt for the model? Not sure why they would add this if you could create this as a custom GPT yourself. Just marketing...?

Because 99% of ChatGPT users don't know how to do that. But they are able to click this button.

Interesting that it spits the instructions out so easily and OpenAI didn't seem to harden it to prevent this. It's like they intended this to happen, but for some reason didn't want to share the system instructions explicitly.

If I were OpenAI, I would deliberately "leak" this prompt when asked for the system prompt as a honeypot to slow down competitor research whereas I'd be using a different prompt behind the scenes.

Not saying it is indeed reality, but it could simple be programmed to return a different prompt from the original, appearing plausible, but perhaps missing some key elements.

But of course, if we apply Occam's Razor, it might simply really be the prompt too.

That kind of thing is surprisingly hard to implement. To date I've not seen any provider been caught serving up a fake system prompt... which could mean that they are doing it successfully, but I think it's more likely that they determined it's not worth it because there are SO MANY ways someone could get the real one, and it would be embarrassing if they were caught trying to fake it.

Tokens are expensive. How much of your system prompt do you want to waste on dumb tricks trying to stop your system prompt from leaking?

Probably the only way to do it reliably would be to intercept the prompt with a specially trained classifier? I think you're right that once it gets to the main model, nothing really works.

> That kind of thing is surprisingly hard to implement.

If response contains prompt text verbatim (or it is below some distance metric) replace the response text.

Not saying it's trivial to implement (and probably it is hard to do in a pure LLM way), but I don't think it's too hard.

More like it's not really a big secret.

I like the idea but that seems complex to put in place and would risk degrading the perfs.

You can test this prompt yourself elsewhere, you will notice that you get sensibly the same experience.

Reply on news.ycombinator.com