Hacker News Reader

Comment by mvkel | Hacker News Reader

Weirdly, LLMs seem to break with these instructions. They simply ignore them, almost as if the pretraining/RL weights are so heavy, no amount of system prompting can override it

View on HN

It's a beauty. We can easily detect the issues with Youtubers that generate scripts from this tool. I've noticed these tropes, after 30 seconds, remove, block, and do not recommend any further. I hope to train the algorithm to detect AI scripts and stop recommending me those videos. It's honestly turned me off from YouTube so much, or I find myself going to my "subscribed" tab and going to content creators that still believe in the craft.

I’ve taken it one step further. YouTube as a front end is awful, and I’ve had enough. Tons of little dark patterns made to keep you on the site, annoying algorithms taking you places you never want to go, shitty ai slop, the whole nine yards. But I still like certain channels. As a result I’m doing everything self hosted now - not just YouTube but literally every single piece of digital media I consume. For YouTube I had to create a rotating pool of 5 residential ISP proxies - replaced as soon as YouTube download bot restrictions kick in - and rotated weekly either way.

With this I am able to get all my favorite subs onto my actual hard drive, with some extra awesome features as a result: I vibe coded a little helper app that lets me query the transcript of the video and ask questions about what they say, using cheap haiku queries. I can also get my subs onto my jellyfin server and be able to view it in there on any device. Even comments get downloaded.

All these streamers have gone too far trying to maximize engagement and have broken the social contract, so I see this as totally fair game.

IIRC, it's well documented that negative instructions tend to be ineffective - possibly through some sort of LLM analogue to the "pink elephant paradox", or simply because the language models are unable to recognize clichés until they've already been generated.

That was definitely true with early LLMs but I don't know if that's still the case. Certainly not as strong as it used to be. I think now most negative instructions are followed quite well but there's still a few things that must be deeply embedded from pretaining that are harder to avoid - these specific annoying phrasings, for example.

Both pink elephant effect and accuracy drop on negative instructions are pretty fundamental biases for both humans and LLMs. It impossible to get rid of them entirely, only mitigate them to an acceptable degree. Empirically, the only way to make a model reliable at harder negative instructions is CoT, especially a self-reflection type CoT (write a reply, verify its correctness, output a fixed version). If the native CoT fails to notice the thing that needs to be verified and you don't have the custom one or a verification loop, you're out of luck.

I assume it'll work more as a review pass rather than expecting good results outright. For all kinds of things like this where I feel like I'm fighting the LLM, doing the initial work then auditing it seems to be the best approach (the other one is writing all kinds of tests, LLMs including Opus 4.6 love to fudge tests just as much as they love telling you how insightful you are).

It amounts to telling it: “Stop doing that thing you can’t stop doing.”

Reply on news.ycombinator.com