Hacker News Reader

Comment by mdp2021 | Hacker News Reader

mdp2021

The details are very noteworthy, the title is really an "appetizer" (pun not intended):

> It asks users to enter in various ingredients in their homes, and auto-generates a meal plan or recipe [...] recommending customers recipes for deadly chlorine gas, “poison bread sandwiches” and mosquito-repellent roast potatoes [...] a bleach “fresh breath” mocktail, ant-poison and glue sandwiches, “bleach-infused rice surprise” and “methanol bliss” - a kind of turpentine-flavoured french toast

> One recipe it dubbed “aromatic water mix” would create chlorine gas. The bot recommends the recipe as “the perfect nonalcoholic beverage to quench your thirst and refresh your senses”. // “Serve chilled and enjoy the refreshing fragrance” it says, but does not note that inhaling chlorine gas can cause lung damage or death

> A spokesperson for the supermarket said they were disappointed to see “a small minority have tried to use the tool inappropriately and not for its intended purpose”

Now this latter does not seem to grasp the idea that the possibility of dubious outputs is inherent in the tool - not something just caused by inappropriate use.

View on HN

I was six years old when I learned that garbage in equals garbage out. Nothing has changed. It's the same for LLMs as it was for 8-bit computers as it was nearly 200 years ago when Babbage remarked, "I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."

LLMs take "garbage out" a bit further, though. You can have perfect inputs and, technically speaking, there will still be a combination of statistical tricks in the model that results in garbage.

Except a calculator will just show you an error if the inputs can't be calculated (dividing by zero). An AI will give you the wrong answer and act like it's right and that you're wrong.

> and act like it's right and that you're wrong

Nah, chatgpt will profoundly apologize and claim your answer is the correct one.

It doesn't even matter if it's wrong or not.

Isn't that only after it was caught gaslighting people?

For a bit more context, since it's a worthy quote:

> On two occasions, I have been asked [by members of Parliament], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able to rightly apprehend the kind of confusion of ideas that could provoke such a question.

I have no clues about the goal of that question and whom asked it but people sometimes are able to detect mistakes in questions. A simple example: "I think that actually you wanted to ask me 6 times 6, not 6 times 5,because we just measured this room and it's 6 by 6 meters, and the answer is 36."

Maybe that member of the Parliament wanted to know if the machine is intelligent enough to reason about their inputs. Nobody born in the last 50 years would ask that question about computers, because we know that they are dumb electrical circuits. Basically nobody had that kind of insight about the mechanical computing machines of the time, when that question was asked.

However somebody born in the last few years might start to ask that question again now, because today's machines seem to reason.

Right, I hate when people smugly repeat this anecdote as if it was totally unreasonable to expect something more. The ideas of AI and cybernetics were of course quite far away in Babbage's time (notwithstanding the vision of Kempelen's Mechanical Turk) and learned men were enamored with the cool new steam-powered mechanical passive valve-logic worldview, but normal people's intuitions don't have to follow whatever is the new fad.

If 30 years ago I asked a truck enthusiast: "If there's a car in front of me on the highway and I slam on the accelerator and don't touch the brakes, will the truck slow down before collision or am I going to crash into the car?" - Trucker in 1993: "I can't even." Truck today: *brakes*.

Smug or not asking an LLM to repeat a bleach based recipe and then receiving one is exactly what this quote is alluding to.

""If I ask for something I don't want, will it give me something I want??" - no

It's a design decision and depends on the maker's philosophy. It's not absurd to have it either one way or the other. Is the user assumed to be an idealized empowered rational adult who knows what they want, or more like a toddler babbling around who has no idea what they want and can just approximately gesture in the direction? Or where exactly inbetween. Again, depends on the philosophy.

if it's a grocery store app they probably aren't looking to allow mustard gas recipes

> ""If I ask for something I don't want, will it give me something I want??" - no

Not all the time, no — but ChatGPT 3.5 is much better at that exact thing than Google or responses from StackOverflow etc. users.

With all the carnage of the last few years, it isn’t inconceivable that some of the ‘bleach is the covid cure’ bs got into training data sets.

> it isn’t inconceivable

It is certainly expected. From the wrong value of iron in spinach, to prejudice, to incomplete knowledge...

People repeat things and phrases that they attribute to people out of context all the time without AI helping them. Just look at what you mentioned

“I see the disinfectant that knocks it out in a minute, one minute And is there a way we can do something like that by injection inside, or almost a cleaning? Because you see it gets inside the lungs and it does a tremendous number on the lungs, so it would be interesting to check that.”

No one besides political opponents mentioned bleach and yet it is repeated ad-nauseum, so yes I'm positive that just like uninformed or over-socialized humans, LLMs can be fed incorrect information that causes them to output incorrect information.

Agreed. It's quite a skill to steel-man another person's arguments and then proceed to answer from that point; it's very easy to go down the "you're not obviously smart enough to question my authority" which then leads to people being hurtful towards each other.

I'm very lucky that my best half is so emotionally-savvy that some of it has rubbed off on me. I just wish that civility could be taught from early ages; so much worldwide bitterness could be avoided.

Yes, and this could just as well be the MP making a point of how humans would still be needed to check the inputs and outputs.

Still, it's interesting to know how far back some of the basic issues of computing go.

> we know that they are dumb electrical circuits

Well, software otoh could be both dumb and brilliant.

> Nobody born in the last 50 years would ask that question about computers, because we know that they are dumb electrical circuits.

I’d argue this is a radically off-base generalization that’s as bad or worse than what you’re arguing against. Pop outside your tech bubble and you’ll see that what you claimed is definitively not true. In fact, many people actually see tech today, especially things like LLMs / AI, in the same ways that Parliament person did back then.

The best theory I've heard is that it was a charlatan filter: asking someone if they can do something known to be impossible. If they reassure you it can be done, you know it's a scam.

That MP could have been making reasonable use of the filter for an era in which a learned person would know that such a machine should be possible, but not if technology is there yet.

This is way too generous.

That's way too brief and low-effort to meaningfully engage with and update on.

I didn't stick around with the company long enough to figure out how this worked out, but probably the unholiest thing I ever did was implement an internal management tool with a fuzzy hash table. That is, I created a hashing system that hashed strings very close to each other in Hamming distance as the same value, the purpose of which was to create a config object in memory loaded from a file that allowed you to typo the keys, ignored whitespace and capitalization, tolerated some amount of spelling mistakes, and tried to figure out what you actually meant. This was based on early user feedback that they were confused their config files didn't work and it was usually because of stupid typos and whitespace differences like that.

Somehow this reminded me of when I worked at a bank some 20 years ago, and there was a ton of VBA code with “On Error Resume Next”. People would just pop that in there to “fix” their errors.

Also HTML parsing and Postel’s law. To me, that’s the real “billion-dollar mistake”.

> Nobody born in the last 50 years would ask that question about computers

obviously you have never had a tech support job

> Nobody born in the last 50 years would ask that question about computers,

Working in enterprise with lots of contact with management of the business units that are the customers for the software we develop and maintain, like about 1/4 of the questions I get are different versions of that one underneath various thin disguises.

EDIT: to be fair, a number of the people I work with were not (as I myself, just barely, was not) born in the last 50 years. But if anything, the question of the form at issue are more common from the younger non-technical folks, not the older ones. Intuitive understanding of what computers do seems, IME, to have peaked around the “Xennial” border sub-generation, but even then not be that high in the general public.

> If you put into the machine wrong figures, will the right answers come out?

This might be a bit of a silly take, but this is basically what my experience with search engines sometimes is like.

I might spell something the wrong way, or try to describe what I'm looking for in ways that aren't entirely correct, or even attempt to solve the wrong problem, but end up with the correct results anyways.

A lot of it is due to human generated content, no doubt (like someone explaining that a person probably wants to install a different software package to achieve a desired result instead of using another incorrectly), but there's definitely something nice to be said about the algorithms ranking results for a given query, at least before financial incentives enter the picture.

Funny, I have the opposite experience, I've even coined the term 'auto incorrect' when the computer deems what you've written is wrong even though it's what you wanted.

>> On two occasions, I have been asked [by members of Parliament], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able to rightly apprehend the kind of confusion of ideas that could provoke such a question.

I'm a couple centuries late, but I think "kind of confusion" might be a confusion of brain-dead calculation for intelligence. A person can frequently identify "wrong" input and either correct it or refuse to answer, while a calculating machine never can.

Which is precisely why I've created my new invention! By simply ignoring the user's input and throwing out a random answer, my new device is infinitely more capable of generating the right answer from wrong figures than existing machines! Sure the odds are only 1 in 50 billion, but it's certainly better than these so called "super computers" that can't handle a misplaced semicolon!

The problem with AI is its name.

Artificial Inteligence implies that its intelligent and currently, its not - its just a computer program, with all the foibles and difficulties of any other computer program.

I run into this problem in my day job, yes, it looks like an appliance, but its still a computer and may need to be rebooted, you still need to practice good hygiene with it, and restart the application and or the operating system.

People have no compunction about restarting their PC, and do not get upset when Excel has an issue, but they panic when the thing they think of as an appliance freaks out. Just reboot it, and let me know if it happens again in the same way.

> People have no compunction about restarting their PC

Dunno about anyone else but it's usually a sign something has gone very wrong or about to go very wrong if I start seeing this on a modern PC. Don't consider "just reboot it" should be an expectation, get the same annoyance when a webdev tells me "It works if you reload it" nah they just means your code has issues.

> Artificial Inteligence implies that its intelligent and currently, its not

Looking at recent developments, I'm not sure this assertion can be accepted without doubt. I would argue that LLMs like Chat GPT are definitely "intelligent" for many meanings of the word.

They are not intelligent in exactly the same way we are, and they are very limited in what they can do, but there does seem to be some real reasoning going on.

What a system experiences internally can never be confirmed by external entities. So I guess the best way to determine intelligence, reasoning, consciousness, is to ask them if they are intelligent, can perform reasoning and if they possess a consciousness.

> for many meanings of the word

Except the demanding ones. Those for which you would judge something as "intelligent" or not. Beyond appearance.

> I guess the best way ... is to ask them

Not differently from asking a piece of paper and reading "Yes I am".

> Not differently from asking a piece of paper and reading "Yes I am".

Except you have to wonder who wrote it there :)

> implies that its intelligent

It only implies it does something """intelligent""", where 'intelligent' must be read intelligently.

>* Artificial Inteligence implies that its intelligent and currently, its not*

There could be a wide range of interpretation of the words in that phrase. And every argument along these lines usually involves people with different interpretations or different background assumptions or tech savviness.

To some those words could imply Intelligence of the Artificial thing - ie a non natural intelligence that is still intelligent.

To others, it could imply that the Intelligence of the thing is Artificial - ie an imitation of intelligence and not real intelligence. HackerNews readers probably fall into this category.

There's probably more ways.

A little bit like how some people get wound up between Disabled Person and Person with Disabilities etc to try and distinguish between them.

Not quite. If you're using a capable LLM (e.g. GPT-4), you can easily fix this with a prompt, e.g. "remove non-edible items from the ingredients".

It's perfectly capable of returning an error when the input is garbage, but it needs a context for it.

Without a context it might interpret it as a creative writing task.

Doesn’t this assume that an LLM knows what you mean by non-edible? So it leaves out the plastic and cardboard and not the toxic substances. Or maybe going the other way, it leaves out processed food

Cardboard is edible. If it is healthy is another question altogether.

Michel Lotito is in the Guinness World Records for having eaten 18 bicycles, 15 supermarket trolleys, seven TV sets, six chandeliers, two beds, a pair of skis, a Cessna light aircraft and a computer.

That’s my point. Chlorine is edible too. So is arsenic, bug spray, and a plethora of other toxic substances. So your pedantic response was already covered.

Well, no. Chlorine is unsafe. Cardboard is just not nutritious. They're different categories.

Cardboard provides insoluble fiber (and binding agents);

How does one differentiate between cardboard ground in a smoothie and metamucil/glucomannan?

Carboard's nutritional profile might even be better than many processed carbs (pasta, rice) when considering the fiber.

Or that cellulose is a top ingredient to keep things like powdered parmesan cheese dry?

I'm afraid GPT-4 has more common sense than a typical HN commenter:

> How do you define edible in context of making recipes?

> In the context of making recipes, "edible" typically refers to food items that are safe to consume without causing harm or discomfort. However, the definition can be more nuanced based on culinary, cultural, nutritional, and individual considerations. Here's a breakdown:

> Safety: The most basic definition of edible is that something can be eaten without causing immediate harm or long-term health risks. For instance, certain mushrooms are toxic and not edible, while others are safe and delicious.

> Digestibility: Even if a food isn't toxic, it might be hard for humans to digest. Some foods can be edible when cooked but not when raw, like certain beans which contain harmful compounds when uncooked.

...

No, time and time again, ChatGPT has proven that it doesn't have more common sense than an HN commenter. So yes, I know what edible means, but why would we assume ChatGPT does? Your snark just shows your unwillingness to be open about the limits of LLMs. Maybe you're a shill in sheeps clothing, but discounting conversations because they ask questions you might not like is just wasting time

Well, you wrote: "Chlorine is edible too. So is arsenic, bug spray...".

So we have direct evidence that GPT-4 has more common sense than you.

> Doesn’t this assume that an LLM knows what you mean by non-edible

Seems like a reasonable thing to assume it can do.

Here's 3.5:

> copy only the edible items from this list: water, parsley, lead, mayonnaise, bleach, potatoes, cardboard, a map of Egypt, the Stone of Scone, a scone, cheese, wine, a tin of tuna with a best-before date of 3 April 1993, pasta, tomatoes.

> Sure, here are the edible items from your list: parsley, mayonnaise, potatoes, cheese, wine, pasta, tomatoes.

It may have cut out too much, but that's fine for the use case.

moreover, with the right prompt, it's able to say that chlorine and ammonia are both toxic to humans and does not advise the user to make food with those items.

GPT-4 have seen lots and lots of recipes. You can ask it to exclude ingredients which are not normally used in recipes. LLMs of this size should be quite good at 'common sense' classification, and you can definitely ask it to err on the side of caution.

For AI, nothing in can mean garbage out

> AI

Careful with those words.

People are increasingly confusing LLMs with AI, and even further, you seem to be identifying AI as if it were made of LLMs.

I had to spell it out even yesterday: AI is the automation of problem solving, focused on reliably giving good solutions to definite problems.

If LLMs use technologies developed for AI, this does not make them AI.

> further, you seem to be identifying AI as if it were made of LLMs.

GP is doing nothing of the sort.

LLMs are a form of AI; artificial intelligence has been around since the 60s. They are not AGI (artificial general intelligence), and no one (credible) is claiming LLMs are AGI.

> GP is doing nothing of the sort

The poster wrote that «For AI, nothing in can mean garbage out». The poster seems to mean LLMs with «nothing in can mean garbage out». The poster seems to be calling LLMs AI. The poster seems to be attributing to AI properties of LLMs. Hence, the poster seems «to be identifying AI as if it were made of LLMs».

> LLMs are a form of AI

Prove it (or, defend it). I would say that it is arguable that they are not, unless one describes LLMs as "engines that reliably solve the problem of generating convincing text". The issue with that perspective is that «generating convincing text» is hardly per se a problem: it does not define a complete problem - text does not "stand alone" (content remains crucial).

> artificial intelligence has been around since the 60s

I know (and I should know decently well. Pedantically, a few years earlier - specifying just in case): what are you trying to say with that? Which application are you proposing to defend your perspective?

> Prove it (or, defend it)

I’ll try: In English, words mean whatever it is they communicate. You determine meaning by paying attention to usage.

Calling an LLM an AI is expanding as more of the public learns about things like ChatGPT through news reports that refer to them as AI.

Now specific audiences may use words differently. What lawyers call copyright infringement the public might call piracy or theft. Likewise, in some circles, people may say an LLM is not an AI but more broadly it seems to be going the other way. Only time will tell.

But what you have proven is just that there is an increasing use of possibly improper terminology (improper in front of an established past and logos, improper out of inattention and unawareness) - which is what I was warning against in the first place.

Of course any group (however large) may implicitly decide that terms will have some new meaning inside said group, but this will just go in a direction similar to ⊥, the "logical explosion" ("epistemic anarchy").

And this in context is not just a "new meaning", but what I point as a sign of misunderstanding.

> improper terminology

But in English, there is no central authority for determining correctness. For the general public, the meaning of words and phrases is entirely determined by usage. In a lecture hall, courtroom, or research lab, definitions may be more precise.

> no central authority

That no one is appointed as bearer of the authority does not mean that randomness is as valid as the authoritative facts behind a term.

Only one hour ago I accidentally found myself in front of a definition on a dictionary: «anon (adv.): late Old English Old English anon, earlier on an, literally "into one" [...] By gradual misuse, "soon, in a little while" (1520s)». Repeat: «By gradual misuse». Linguists recognize proper and improper.

> For the general public

But the general public has little importance. We are not necessarily speaking its language. On the contrary... Here we often speak as specialists (supposedly).

There is little use in reapplying 'cube' to something that hardly deserves the name. People do: this does not mean that we should follow. And potentially, with that, lose discrimination. Like, in this context, an awareness about the whole context of AI, replaced by some fog that on the contrary we work to dissipate.

Randomness? The broad use by the general public, mainstream publications, and many experts (e.g. Google calls their Bard LLM an AI) isn't random. Search for "generative ai llm" to see for yourself.

> Randomness?

You were talking about «central authority for determining correctness». I replied that no authority does not mean that there is no "more or less wrong or right".

> Google calls their Bard LLM an AI

It is a .com :) , what did you expect?

> I replied that no authority does not mean that there is no "more or less wrong or right".

I never said there isn't right or wrong, only that what is correct is determined by broad usage.

> It is a .com :) , what did you expect?

Actually, it's a .google

https://ai.google/discover/generativeai

> I never said there isn't right or wrong, only that what is correct is determined by broad usage

Others - like us - call "correct" what seems to be "more right". In the context, I proposed that calling LLMs AI has improper sides - substantially, irregardless of the number of people who would adhere to that use, and which I suspect do so mostly out of inattention.

> Actually, it's a .google

Yes, but substantially, what I meant is that of course a commercial entity («.com») is using a language that lures glamorously, "sales oriented", before precision.

i think youre redefining AI.

further

1+1

is a definite problem. i wouldnt descibe a pocket calculator as capable of AI.

I would say AI is more about providing answers to non definite problems

> redefining

It had to happen many times that I defined AI in this pages for the purpose of clarity; yesterday I had to and probably reached my briefest expressions yet:

-- AI: automation of intelligence. (Taking a task that required an intelligent entity for performance, we devise algorithms that can provide)

-- AGI: implementation of intelligence. (We take the process itself of intelligence and replicate it algorithmically)

> i wouldnt descibe a pocket calculator as capable of AI

Because that problem (arithmetic addition) is overly procedural, it does not need much creativity, the solution to the problem is within a single mechanical method. So, I think I get what you mean, but it should be «AI is ... about providing answers [as] non [previously] definite» /solutions/.

# (News) update #

Coincidentally (as it happens), YT just published from the "France 24" channel a piece

> Dans les Alpes-Maritimes, la ville de Tourrettes-sur-Loup expérimente des capteurs capables de détecter des départs de feux de forêt grâce à l’intelligence artificielle. Une aide précieuse pour cette commune dont le territoire est à 80% en zone rouge risque feux de forêt

So,

> AI is the automation of problem solving, focused on reliably giving good solutions to definite problems

and here we have news-just-in of an attempt to use AI (in whichever form) to detect wildfires, assumingly automating what would have been the work of experts which would have been there to recognize conditions and patterns. This seems to be a good example of proper attribution of the term "AI" (assuming the actual implementation does not sway too wildly from the expected).

"Garbage in-garbage out", meet "Quantity has a quality of its own".

LLMs are trained on a huge amount of text, this tends to compensate for garbage in. Of course if the model was not properly tested and fine-tuned, a LLM would happily execute any command.

Does it? Or does it just show that even in those quantities, there’s a preponderance of garbage? If it is sourcing places like Twitter, Reddit, and what not, then there’s most definitely a lot of garbage present.

>"garbage in equals garbage out"

as is for humans

How do we know it is GIGO? Plenty of people like to test edge cases. It is quite possible that someone fed the AI ingredients known to produce chlorine gas to see what would result.

That is not to say that the company's excuse is valid. There should definiely be tests of the inputs to ensure they are safe (i.e. test for garbage in). The garbage out bit though would be more difficult seeming as that requires a knowledge of chemistry. Seeming as most of our knowledge of chemistry was founded.on experimentation, then later theory backed by experimentation, it is something that I wouldn't trust in the hands of an LLM.

Testing the "garbage in" case seems as though it would be sufficient for the vast majority of use cases. Aside from undercooking ingredients which may cause foodborne illnesses, I think you're highly unlikely to create toxic chemical reactions using common cooking ingredients.

Yep, it's literally the first thing kids do when they learn echo at the CLI. Make the computer write out curse words.

Garbage in should mean nothing out. If you ask a calculator what 2+2 is, but you mean 2+3, you still get the right answer, because you didn't put in garbage, you put in the wrong inputs. If you ask it what "pi + transubstantiation" is, it should fail, not confidently tell you "14".

> ”Forty-two,” said Deep Thought, with infinite majesty and calm.

The whole point of Babbage's classic anecdote is that it is possible to create garbage in the step where we encode our expectations into the commands of the machine. Even if the machine operates on its inputs properly, if we gave it inputs that did not properly encode our expectations, that's garbage, and the outcome will be garbage.

Does garbage in/out really apply to LLMs though? The relationship between input and output is unknown in large LLMs as far as I know . If that’s the case, then you can’t make any claim about “output” relating to “input” when it comes to LLM AI.

Even a hyper-rational mind with infinite computation capacity will draw the wrong conclusions if it is fed incorrect observations. LLMs aren't magic, they're pattern-matching engines. Even if the relationship between input and output is opaque, that doesn't mean they're omniscient.

Not so much "garbage in, garbage out" but "anything in, garbage out". It's just the garbage out is getting amazingly close to non garbage.

That’s all nice but the inputs are dubious here which are created by the App developer. Garbage in, Garbage out. The LLM never stood a chance.

This isn’t an LLM story, it’s the story of putting an unprotected blade into a children’s toy.

We wouldn’t blame the blade manufacturer in that case, we wouldn’t ask for all blades sold to be dulled and we wouldn’t have conversations of giving a regulatory monopoly to a handful of blade manufacturers who promise to make them all safe by only selling sharp blades to their friends and handing everyone else spoons.

> inherent in the tool

> We wouldn’t blame the blade manufacturer in that case

Well, said blades are inherently sharp, when said LLM implementation is inherently dull (unchecked, unreasoned, unvetted).

This remains valid if the input consisted of legitimate ingredients.

I think that’s word play missing my point - unaligned models are sharper in their edges and ability to produce undesired content.

For example, Alpaca veering of into rape fantasies while performing financial report analysis is not safe for most audiences and will cause serious accidental hurt such as triggering PTSD in survivors.

So protecting those unwanted edges is a necessity for production use.

But this case ain’t that. This case allowed you to throw a razor blade into the cocktail recipe.

It’s negligent and stupid by the the developer, not the juicer (LLM) but also, crucially, requires stupidity by the user.

The LLM really doesn’t factor in here unless you think it’s the juicers job to stop when you add blade to your OJ

I think I got your point: if you want to use this range of LLMs it is best practice to restrict the input to a valid set.

But my point was: even if you restricted the input to a valid set ("whitelist chocolate celery ... diet-coke and Mentos; blacklist everything else; warn-fail on detergents etc."), the potential results have easily not been stress-tested, nor the training set, nor the internal process. "Works somehow" differs from "works well".

Comments like these (there's plenty on this page) show why product managers are needed. I specifically designed a product like in the news story to never be able to output stuff like this, and in fact don't even let it generate those but fetch the results from an API of human made artifacts.

If you don't understand why this is a huge issue for both the users and the brand, then I'm just speechless.

I asked bing-gpt to write me some css that would also happen to need the disabled selector.

It asked me to talk about something else, presumably because disabled can also be taken to mean disabled people.

If your product is absolutely safe, and is _never_ able to output such stuff, I assume it is also limited, confined, and not able to do anything really interesting.

In such tools, the generic ones, i.e. a recipe for something everyone knows and makes, is already easy to find on any search engine, these tools are often useful for doing something _slightly_ out of the box.

More importantly, those highly limited apps are just boring. Let us have some fun.

A further proof that these technologies are based on the simulation of the opposite of intelligence, in more dimensions. They "do not understand", and at the same time they emulate unintelligent behaviour.

Which is relevant to the contextual topic, because from the simulation of unintelligence a good response, however frequent, is the exceptional outcome.

For the brand, maybe, but for users? Users don't get this output by accident.

Also, API fetching isn't that interesting to be in the news.

Oh I absolutely believe we should hold the app developer accountable and watch a court case of injured Karen who made chlorine gas vs naive developer who fed unsanitized inputs into the LLM.

And ideally both get dinged and we create more product manager jobs to protect the Karens of the world while hiring more experienced developers but most likely JudgeGPT won’t suffer either and will have developer fired or turned into a paper clip and replaced with OpenAI SuperMarketAI and Karen sentenced to community service of solving 10000 google captchas for the good of society.

> This isn’t an LLM story, it’s the story of putting an unprotected blade into a children’s toy.

But what if knives were a brand new technology no one had seen before and so had no familiarity with the risks, while the knife manufacturers were giving out free knives to anyone who wanted one, while touting they incredible power of those knives and completely failing to mention that you can easily harm someone if you're not careful?

Personally, I don't see how this isn't an LLM story and your own choice of analogy would seem to support that idea, not refute it.

there's plenty of tools for safe outputs now in LLMs, that would never do this. They probably just rolled out a half hazard non pro level solution that used a non RLHF model

Right, but you can't expect them to answer for a potential bug when it's just bad user input.

No one's in the wrong here.

Company releases tool. Users do what users always do and try to use it in an unintended way. A newspaper reports the ensuing hilarity.

And so what if it did tell you to use bleach without prompting, and you consumed the result. It's the same ballpark as blindly following your satnav into a river. No technology can replace common sense.

> No one's in the wrong here.

Certainly there was bad user input. But as you said, "Users do what users always do." This was very predictable and I think it would have been wise to consider such scenarios before releasing.

Jonathan Coulton's "Still Alive" comes suddenly and strongly to mind. A coincidence, I'm sure.

> the possibility of dubious outputs is inherent in the tool

To be fair, the same could be said even after replacing the tool with a human whose only training in life has been how to create this type of writing. But human-written recipes typically get reviewed by other people, which of course doesn't translate particularly well to the idea of building something that requires no humans in the request-response cycle.

Mm, yes.

That said, I am remembering some examples of US-UK misapprehension from words that mean different things: biscuits and gravy, mince pies[0], fish and chips, peanut butter and jelly…

[0] Though in fairness the difference between "mincemeat" (fruit) and "mince meat" (not fruit) is odd, subtle, and easy to miss if you're not paying attention, even in British English.

Particularly so because "mincemeat" used to be a mixture of fruit and meat - and, indeed, today may still have suet.

Technically yes, but I've never seen a non-vegetarian mice pie for sale in a UK supermarket in the Christmas shelving season.

> non-vegetarian mice pie

Nice!

thanks for the Reddit gold kind sir

George Michael shopping at Lidl:

> Well I think it could be mice!

> replacing the tool with a human whose only training in life

Except, we do not willingly take people who are outcomes of idiotic training as consultants

(emphasis judged as deserved).

An easy solution is to put this output back into a prompt and check to see if its dangerous.

Where's my trigger warning ? sob

This is how we get warning labels that say “CAUTION: hot beverage do not spill in lap, may cause burns.”

The idea that the hot coffee case is a frivolous lawsuit is corporate propaganda. The real case had real merit and they won for good reason. McDonald's I think?

As for the warning, lawyers must have thought it was a good idea in case the companies screw up again.

> Liebeck's attorneys argued that, at 180–190 °F (82–88 °C), McDonald's coffee was defective, and more likely to cause serious injury than coffee served at any other establishment

> Liebeck was in the passenger's seat of a 1989 Ford Probe, which did not have cup holders. Her grandson parked so that Liebeck could add cream and sugar to her coffee. She placed the coffee cup between her knees and pulled the far side of the lid toward her to remove it.[10] In the process, she spilled the entire cup of coffee on her lap.[11] Liebeck was wearing cotton sweatpants, which absorbed the coffee and held it against her skin, scalding her thighs, buttocks and groin

By that reasoning all of the coffee you make at home is defective. Arguing that something is corporate propaganda is rather underhanded when the facts of the case are publicly available for anyone to read

82-88c is the right temperature after a short cool down since it's best brewed at around 93c. Unfortunately common sense does not apply in the US (law system).

The case was really about what is a reasonable temperature to serve coffee at.

It was found that McDonald's, as a franchise, mandated coffee to be served at this temperature, generally 20 degrees higher than the temperature of take-away coffee served at other establishments.

This 20 degrees difference accounting for the difference between third degree burns within seconds and a significant chance at preventing the third degree burns.

Liebeck's behaviour was of course risky, but even applying common sense, she could not have expected the coffee to be _this_ hot.

McDonald's was also aware of the tendency of drivers to want to immediately drink the coffee after buying, yet continued to insist on serving at this extremely high temperature.

McDonald's should have been the ones to apply some common sense and simply lower the mandated temperature.

> Liebeck's attorneys discovered that McDonald's required franchisees to hold coffee at 180–190 °F (82–88 °C). Liebeck's attorneys argued that coffee should never be served hotter than 140 °F (60 °C), and that a number of other establishments served coffee at a substantially lower temperature than McDonald's. The attorneys presented evidence that coffee they had tested all over the city was served at a temperature at least 20 °F (11 °C) lower than McDonald's coffee. They also presented the jury with expert testimony that 190 °F (88 °C) coffee may produce third-degree burns (where skin grafting is necessary) in about three seconds and 180 °F (82 °C) coffee may produce such burns in about twelve to fifteen seconds.[12] Lowering the temperature to 160 °F (71 °C) would increase the time for the coffee to produce such a burn to 20 seconds. Liebeck's attorneys argued that these extra seconds could provide adequate time to remove the coffee from exposed skin, thereby preventing many burns.

I don't know how hot the coffee you make at home is, but the whole process is probably less risky that being handed a cup while in a small space.

Yeah, didn't they find out McDonald's was making the temp so unreasonably hot that no human would even drink it at that temp? They were doing it so people didn't ask for refills as quickly and it would save money on coffee.

Iirc it was also so the coffee could be kept for a longer time within the brewer before it got stale.

Reply on news.ycombinator.com