>Comments should be passed to the model with clear role boundaries that prevent them from being interpreted as system-level directives.
Well, such clear boundaries would solve lots of problems. But those don’t exist, do they?
Well, such clear boundaries would solve lots of problems. But those don’t exist, do they?
They require being able to transorm the output to something symbolic, but this YouTube feature necessarily has to output free-form text, derived directly from the comments..!
What would actually prevent the "attack" is for YouTube to not turn markdown from random LLM outputs into actual links.
In general, those patterns seem applicable only to a limited amount of cases, I think that they prevent much less than 99.9% of the attacks.
I don't understand how these systems are passing the bar. You would have been fired for trying to railroad something like this into production 10-15 years ago. What happened?
Personal finances aside, that's no way to run a business. Torching your brand, alienating your users, and pissing off your customers is a well known path to ruin. [edit] Even if it results in some temporary windfall--is the thesis that the windfall will be so big they no longer need users or customers? It eludes me completely what the companies that are building this trash now are hoping to achieve. It's especially galling that publicly traded companies are doing it. It's one thing for a startup to blow a bunch of venture capital on a speculative, half-baked product idea. Great risks sometimes yield great rewards. Usually they don't. Founders and VCs knowingly and willingly sign up for those risks. It's a totally different story for public companies.