Hacker News Reader

Comment by Aachen | Hacker News Reader

> When customers began experimenting with entering a wider range of household shopping list items

So it works as intended: ask it to generate a recipe involving chlorine cleaning products and it does so. Which specific ingredients the user input is not mentioned in the article, presumably because then cause and effect are obvious to anyone.

Next week on the Guardian: hammer used to smash own thumb, consumers calling to remove from shelves. Terms of service said it was for 18+ users only but age not verified upon checkout!

View on HN

The expected answer should be, those ingredients are not edible, I can't give you a recipe that includes them.

Maybe they can fix it with a prompt that makes the model check if the ingredients are safe. Chances are that every prompt can be circumvented but they can probably find instructions for self poisoning somewhere else with less effort.

What about quicklime (calcium oxide)?

It's used here in Argentina to make pumpkin "sweets". (I'm not sure the correct translation.) It's a lot of 1 inch cubes of pumpkin, boiled for some time in sugar syrup. [1] Without treatment the cubes will disintegrate while boiling. If you put them for one day in water with quicklime, the cubes get a hard wall and they keep their integrity while boiling, but the interior is soft.

Drinking the water with the quicklime is dangerous and it's discarded. Eating directly the quicklime is even worse. But quicklime is useful for cooking.

[1] Some random recipe I got in Google (autotranlation) https://cookpad-com.translate.goog/ar/recetas/94999-zapallo-... (spanish) https://cookpad.com/ar/recetas/94999-zapallo-en-almibar

https://en.wikipedia.org/wiki/Nixtamalization is done using limewater and actually vital to sustain life on a maize diet. But it should come with a warning.

Similarly, ramen noodles are made with a bit of sodium carbonate.

Sodium Carbonate is edible. It's used in a few recipes, like a strong version of baking soda (sodium bicarbonate) https://en.wikipedia.org/wiki/Sodium_carbonate#Food_additive... I'm not sure how dangerous is a teaspoon of pure Sodium Carbonate alone, because it's very basic (anti-acid), but mixed with some weak acid it's used to make the fizzy power that is inside some cadies https://en.wikipedia.org/wiki/Sherbet_(powder) (The proportions are important, don't try silly stuff at home.)

A teaspoon of quicklime in your mouth first reacts with water to produce a lot of heat that will burn you, and then the result is very basic (anti-acid), and make new burns in a different way. Definitively don't try it at home.

You can also make ramen with sodium hydroxide. Sodium hydroxide is also used in a couple of recipes (e.g. pretzels). A little bit won't kill you, but I wouldn't suggest a teaspoon full of it.

> Maybe they can fix it with a prompt that makes the model check if the ingredients are safe.

Just sanitize the input and remove products belonging to categories that aren't edible.

Tampons aren't great in salads, and toilet paper is not a good garnish.

> Maybe they can fix it with a prompt that makes the model check if the ingredients are safe. Chances are that every prompt can be circumvented but they can probably find instructions for self poisoning somewhere else with less effort.

That seems like a job for a database, not a language model. Every problem is not a nail.

Well, the input didn't say that the recipe was intended for consumption by humans.

This assumes that someone trained it on non edible items. That is a reasonable assumption, but it is also easy to see how the product developer overlooked it.

TBH, there's often a huge chasm between product planning and real world users at companies and organizations like this.

I believe the issue here is that, as a user, I expect the system to "do no harm". Sure, once you give bleach as an ingredient then all bets are off (although a good system would filter it out), but it highlights the deeper issue that some foods could be bad for my health and the system is letting them through.

Unheated castor oil contains ricin, and even heated oil can cause contractions in pregnant women. Raw chicken can give you salmonella. A working recipe system should therefore never return "chicken sushi on a bed of vegetables with a touch of castor oil" without some SERIOUS warnings, but it's clear that this system may do so. Putting such a system in the hands of unsuspecting users is bad and they should feel bad.

And going further, it demonstrates that the model doesn't actually understand food or cooking, and is just smashing random words together.

It does further the point that LLMs are unable to understand, infer, deduce, or reason.

LLMs are toddlers who are capable of stringing words together that make some sense most of the time. It’s not artificial intelligence.

OK, ... so you rather justify a recipe app accepting chemicals as inputs to blame a news site for their reporting than to identify the problem?

Edit: removed a part about The Guardian.

I'm not sure what the ability to blame the Guardian has to do with anything, I'm basing the above comment solely on this submitted article and don't really know the Guardian as an organisation. I have no beef with them, if I'm reading your tone correctly.

As for allowing to input chemicals: garbage in, garbage out. We accept that from tools (and some countries from guns) but not from software. Maybe that's how it should be, but I find it an interesting discrepancy. In general, as a hacker, I'm happy if a tool lets me use it for unintended purposes (such as humor in this case)

OK, I took The Guardian part out but I think the rest is still applicable. Why is the reporting problematic but not that this problem happens in the first place?

Because there's no story. It's software that generates recipes from ingredients you provide. If you provide inedible ingredients you get an inedible recipe. What did you expect would happen?

The subtext here is journalists who are dead scared of a tool they know will replace them. So they have to denigrate it.

Expecting a tool meant to generate cooking recipes to reject things that aren't usual ingredients of food is not surprising. This tool violates that expectation.

Even trivial things like input fields on web forms have a type attribute that lets developers specify which type of content is expected and valid. (If you're not familiar with that, have a look here: [0]).

For things that might endanger the health of customers, we can expect better.

[0] https://developer.mozilla.org/en-US/docs/Web/HTML/Element/in...

I think that expectation is only violated if you get unusual/dangerous outputs from innocuous inputs.

Just sounding this out but I think the expectation you are responding to is an expectation of an appropriate level of safety, which is more context dependent. For example, do any of the generated recipes actually pose a danger, in that they will participate in harming a human? My hunch is no, this would not present the same kind of danger the 'tide pod challenge' did and would never lead to unusual adverse outcomes (anyone can find out about an allergy from a new ingredient in a recipe for example).

Just exploring the topic here though. I think the perspective that a 'meal planner' should never produce plainly hazardous meal plans is perfectly rational.

if I tell a piece of software to generate a bleach casserole, it should generate a bleach casserole, because I know what I want better than the software does.

anything less would be like clippy telling me Microsoft Word refuses to write my planetary doomsday ransom notes because Microsoft Word is 'meant to write nice things', or my car refusing to exceed the speed limit because it is 'meant to drive legally'

maybe I'm just an alien who's capable of drinking bleach, the app doesn't know and it shouldn't have opinions: it should just do what I tell it to do, because it's a "put whatever ingredients you want together" app, not a "this is definitely safe and healthy" app

I think something that's been left unsaid until now is that the implementation is clumsy and downright dangerous, therefore the implementation is totally botched. A supermarket whose job it is to feed people so as to keep them alive is a vital service in the vast majority of economies, and so has the duty to go a little deeper and fine-tune their model or at the very least heavily prompt-engineer their interface layer to avoid such obvious flubs.

There's no point in apologizing for the LLM. Of course it's doing what it's asked to do. That's not the gripe, nor points toward any kind of solution, though.

I agree with the first paragraph but not necessarily the latter. More likely to me seems that the journalists see something that can be construed to be harmful (so they can justify the story), as well as having a high click rate (see, it ends up on HN), and thus good for business

A lot of Guardian readers excoriate the Daily Mail as low quality clickbait and "not journalism", whilst seeming perfectly content to suckle from the teats of the Guardian's own clickbait farm, presumably comforting themselves that there is some sort of intellectual honesty present in the Guardian that is missing from the Daily Mail. I find it quite strange.

Not sure this "us vs. them" stuff / generalising every reader of a website is helping anyone here

Not to be pernickety, but I did say “a lot of” not “all of”…

And in the context of this conversation which was about clickbait on the guardian I felt it was a contextually valid observation. I’m sorry you didn’t find it helpful.

If you are a "technology enthusiast" you will blame everything but the technology for failures. Is this the first time? Definitely not, we all have the running jokes of "it works on my machine" or "damn users if they only" or "it's not a bug but a feature" developer-developed blindness, where the stupid real world keeps interfering with our perfect creations.

You are conflating things. If you sell a product for profit, it needs to be foolproof, so the general public can use it without shooting themselves in the foot. Capable tools have to be gatekept somehow, either through actual hard licenses or soft gates of obscure forums, hidden small communities that don't advertise themselves to the world, so they can do their thing in peace. The trouble comes when the latter kinds of communities build something useful/fun-when-used-correctly and it leaks and people demand to participate, complain that it's too hard and demand it be dumbed down.

So yeah, don't broadcast your stuff to the mainstream unless it's foolproof. But also, let people be, who are satisfied with dealing with the rough-edges stuff.

This is true in many contexts. Think about enthusiasts of weird drugs or other niche communities like sword swallowers. There's no principle of "you are only allowed to do things that are also safe when ignorantly attempted by a random person".

I don't see why it shouldn't. Let me put in any input I want to. There's no story here.

Not really of course. Should make eatable dishes with correct nutritional value, if thats what you are offering.

From the article:

> In a warning notice appended to the meal-planner, it warns that the recipes “are not reviewed by a human being” and that the company does not guarantee “that any recipe will be a complete or balanced meal, or suitable for consumption”.

It is explicit in that it does not offer "dishes with correct nutritional value"

Whether you should be allowed to label/market this as "meal planner", given that you can't label a milk replacement "milk replacement" in many countries, is up for debate, but the software itself is allegedly not dishonest (according to the article, I haven't tried it myself)

It's offering to make a food-recipe, in there is the assumption that it's eatable.

If you are asking to make a recipe of a bike & sand, it should simply say not possible.

Im not saying you shouldn't launch, or call it a fun or experimental tool, but to just disclaimer your way out of things is too easy.

So they are using a software that they know is not fit for purpose. That is at least gross negligence.

Maybe the hammer analogy I used is not great because hitting yourself is obviously bad whereas any of the recipes might be toxic in a way that the user is not aware of, in which case you're right. But the complaint is that it suggests impossible dishes upon inputting ingredients that are obviously inedible and harmful substances, which sounds to me like it is functioning correctly even if this humorous function was unintentional

Who would ask this LLM to make a recipe involving bleach and then actually proceed to make it? Such a person is already at risk of poisoning themselves, the software doesn't suggest it by itself so I don't see how it increases the risk of harm

But the purpose underlining a meal-planner AI is to create meals. If it's creating something that is not a meal it's not doing it's purpose, if the use-case is for meals then there should be guardrails against non-meals recipes, simply because it does not fit the purpose of the product.

It's not a LLM product, it's not advertised as a general LLM that will generate text based on input of shopping items, it's specifically advertised as an AI meal planner and it's not fit for that purpose since it does not guard against non-meals.

That's the issue, the user is supposed to be a layman, not someone that knows and understands LLMs and its limitations.

This thing sounds useless even if you don't put bleach in as an ingredient then. Sounds like some one wanted to write "with AI" on their resume more than any great desire to build something useful.

Spot on. The user asked for recipes using only water, bleach, and ammonia, it's funny but hardly news-worthy. https://twitter.com/PronouncedHare/status/168736440337978982...

> So it works as intended

It’s literally spelled out in the article that a spokesperson for the company said this is not as they intended.

Reply on news.ycombinator.com