Eating Disorder Helpline Disables Chatbot for Harmful Responses | Hacker News Reader

Eating Disorder Helpline Disables Chatbot for Harmful Responses

vice.com

15 pointstoomuchtodo18 comments

The advice isn’t harmful by itself if someone is overweight. 500-1000 less calories per day will result in 1-2 pounds weight loss per week. It’s simple math. Though I can see the bot should probably ask for height and current weight before giving such advice and put a floor on how low is a healthy weight according to their height.

You can imagine that someone who’s reaching out to a chat for people with eating disorders probably needs to hear something else right? The standard flash card that society holds up isn’t cutting it? Or empathy just isn’t your thing?

> 500-1000 less calories per day will result in 1-2 pounds weight loss per week. It’s simple math

You're making a severe logical mistake here. Can you spot it?

Here it is: you're assuming that energy expenditure is constant. Energy expenditure is driven by many factors, such as hormones.

Energy expenditure isn't constant but it can be easily predicted based on body mass and body temperature versus environment. You made a strawman out of the person you replied to, clearly they meant 500-1000 below maintenance (a variable), not 500-1000 below a constant. The severe logical mistake was made by you.

I persons base metabolic rate doesn't vary that much. You will hear people claim to have a slow metabolism, but in reality it makes little difference.

Most people do not do enough physical activity for it to make much of a difference either.

You are naive and misinformed. This is wrong with regards to about everything you just said.

Basal metabolic rate varies dramatically with the amount of lean muscle mass. You can rapidly lose lean muscle mass by having too high of a caloric deficit which is dependent on overall fat %. The less fat, the smaller the deficit.

Some people have metabolic disorders, others have allergies or sensitivities which cause inflammation that halt fat loss. This includes inflammation from leaky gut, or certain common gut-based infections that are almost impossible to get rid of like H. Pylori.

Physical activity's impact is almost neglible compared against diet and muscle mass.

Also a calorie of a food is not necessarily a calorie of the same food cooked differently, the caloric availability changes depending on preparation.

You're right and it works, but people with eating disorders have mental health issues that push them into obsessive behaviours, they need to be trained out of them.

An anorexic could hear that advice and decide that well, 1500 under maintenance is even better right? Not great.

It's going to be near on impossible to create a chatbot that cannot be manipulated into giving an answer which is user wants to hear. Even people are not completely immune to this.

Unequivocally; yes, this is harmful. You are not a doctor, and should not be giving medical advice.

Neither am I but you certainly have not known or worked with people who have done this kind of lifestyle transition.

That level of calorie deficit requires medical supervision because the potential complications are life threatening. They don't say 'less', its 500-1000 calories.

Your kidneys, liver enzymes, and a number of other factors must be healthy before this can even be considered, and you must be using a nutritionally complete supplement each day for shakes and drinking a lot of water while also taking anti-gallstone medication. They run blood tests weekly to confirm this for as long as you are in the program. I've people who I've worked with who did this.

Some people develop gallstones on these low calorie diets, it gets stuck blocking the duct, requires surgery to correct. You can potentially die.

You can have a gallstone attack and other complications if you can't slowly ween yourself off the supervisory diet. Suddenly transitioning from no food to 3,000 calories in a hunger haze can cause your organs to shut down.

Ask a doctor, they'll tell you and set you right.

Original title: "Eating Disorder Helpline Disables Chatbot for 'Harmful' Responses After Firing Human Staff" (compressed due to character limits)

Previous: https://news.ycombinator.com/item?id=36079183

I’m glad the obvious outcome occurred to remind us to rein in our unreasonable enthusiasm.

I wonder if people were trolling it after their announcement that they were firing their human workers.

I would not be surprised if that was the case. It was a monumentally stupid idea to fire the human workers for this particular application. No computer can hope to accurately respond in a way that isn't harmful at least some of the time.

Human's don't have that problem.

> "It was a monumentally stupid idea to fire the human workers for this particular application."

yes it's probably true

> "No computer can hope to accurately respond in a way that isn't harmful at least some of the time."

yeah so far it looks like that's the case

> "Human's don't have that problem."

wait what

fundamental limitation of computers, you run into the halting problem or decidability problem when you have same fixed inputs and multiple different outputs.

human's can handle multiple different meanings given the same inputs and provide a correct response.

> "fundamental limitation of computers, you run into the halting problem or decidability problem when you have same fixed inputs and multiple different outputs. human's can handle multiple different meanings given the same inputs and provide a correct response."

the computer science concepts of the halting problem (https://en.wikipedia.org/wiki/Halting_problem) or decidability problem (https://en.wikipedia.org/wiki/Decidability_(logic)#Some_unde...) aren't relevant to running an eating disorder helpline

> aren't relevant to running an eating disorder helpline

they are when you replace a human's activity with what's effectively a computation engine.

Those have fundamental limitations which are well known, and they did it anyway. Any reasonable professional in a position to advise should have known, there is really no excuse because this is such a fundamental limitation going all the way back to Claude Shannon and Turing. Its discussed regularly in computer science major related undergraduate coursework.

If you are going to use AI, you should be being advised by experts who are knowledgeable in the area especially when there is potential harm as it is safety critical.

As a software engineer with an eating disorder who believes in the right to organized labor; this is awful. The software engineers that built the chatbot should have had QA testing specifically around eating disorder sentiments. The chatbot should never be treated as a replacement for humans with empathy but is a valuable tool for folks who might not be ready to engage with someone. As someone who has been helped by some resources NEDA shared, I really hope they commit to doing better and just frickin pay their hotline workers.

Reply on news.ycombinator.com