Hacker News Reader

Comment by gmerc | Hacker News Reader

That’s all nice but the inputs are dubious here which are created by the App developer. Garbage in, Garbage out. The LLM never stood a chance.

This isn’t an LLM story, it’s the story of putting an unprotected blade into a children’s toy.

We wouldn’t blame the blade manufacturer in that case, we wouldn’t ask for all blades sold to be dulled and we wouldn’t have conversations of giving a regulatory monopoly to a handful of blade manufacturers who promise to make them all safe by only selling sharp blades to their friends and handing everyone else spoons.

View on HN

Comments like these (there's plenty on this page) show why product managers are needed. I specifically designed a product like in the news story to never be able to output stuff like this, and in fact don't even let it generate those but fetch the results from an API of human made artifacts.

If you don't understand why this is a huge issue for both the users and the brand, then I'm just speechless.

I asked bing-gpt to write me some css that would also happen to need the disabled selector.

It asked me to talk about something else, presumably because disabled can also be taken to mean disabled people.

If your product is absolutely safe, and is _never_ able to output such stuff, I assume it is also limited, confined, and not able to do anything really interesting.

In such tools, the generic ones, i.e. a recipe for something everyone knows and makes, is already easy to find on any search engine, these tools are often useful for doing something _slightly_ out of the box.

More importantly, those highly limited apps are just boring. Let us have some fun.

A further proof that these technologies are based on the simulation of the opposite of intelligence, in more dimensions. They "do not understand", and at the same time they emulate unintelligent behaviour.

Which is relevant to the contextual topic, because from the simulation of unintelligence a good response, however frequent, is the exceptional outcome.

For the brand, maybe, but for users? Users don't get this output by accident.

Also, API fetching isn't that interesting to be in the news.

Oh I absolutely believe we should hold the app developer accountable and watch a court case of injured Karen who made chlorine gas vs naive developer who fed unsanitized inputs into the LLM.

And ideally both get dinged and we create more product manager jobs to protect the Karens of the world while hiring more experienced developers but most likely JudgeGPT won’t suffer either and will have developer fired or turned into a paper clip and replaced with OpenAI SuperMarketAI and Karen sentenced to community service of solving 10000 google captchas for the good of society.

> inherent in the tool

> We wouldn’t blame the blade manufacturer in that case

Well, said blades are inherently sharp, when said LLM implementation is inherently dull (unchecked, unreasoned, unvetted).

This remains valid if the input consisted of legitimate ingredients.

I think that’s word play missing my point - unaligned models are sharper in their edges and ability to produce undesired content.

For example, Alpaca veering of into rape fantasies while performing financial report analysis is not safe for most audiences and will cause serious accidental hurt such as triggering PTSD in survivors.

So protecting those unwanted edges is a necessity for production use.

But this case ain’t that. This case allowed you to throw a razor blade into the cocktail recipe.

It’s negligent and stupid by the the developer, not the juicer (LLM) but also, crucially, requires stupidity by the user.

The LLM really doesn’t factor in here unless you think it’s the juicers job to stop when you add blade to your OJ

I think I got your point: if you want to use this range of LLMs it is best practice to restrict the input to a valid set.

But my point was: even if you restricted the input to a valid set ("whitelist chocolate celery ... diet-coke and Mentos; blacklist everything else; warn-fail on detergents etc."), the potential results have easily not been stress-tested, nor the training set, nor the internal process. "Works somehow" differs from "works well".

> This isn’t an LLM story, it’s the story of putting an unprotected blade into a children’s toy.

But what if knives were a brand new technology no one had seen before and so had no familiarity with the risks, while the knife manufacturers were giving out free knives to anyone who wanted one, while touting they incredible power of those knives and completely failing to mention that you can easily harm someone if you're not careful?

Personally, I don't see how this isn't an LLM story and your own choice of analogy would seem to support that idea, not refute it.

there's plenty of tools for safe outputs now in LLMs, that would never do this. They probably just rolled out a half hazard non pro level solution that used a non RLHF model

Reply on news.ycombinator.com