Hacker News Reader

Comment by firefoxd | Hacker News Reader

Well that was fast. Kudos, really neat. Speed trumps everything else. I only noticed the robotic voice after I read the comments.

I worked on an Ai for customer service. Our agent took the average response time of 24/48 hours to merely seconds.

One of the messages that went to a customer was "Hello Bitch, your package will be picked up by USPS today, here is the tracking number..."

The customer responded "thank you so much" and gave us a perfect score in CSAT rating. Speed trumps everything, even when you make such a horrible mistake.

View on HN

"The customer responded "thank you so much" and gave us a perfect score in CSAT rating. Speed trumps everything, even when you make such a horrible mistake."

I think not everyone would react the same way. For some calling each other bitch is normal talk (which is likely, why I it got into the training data in the first place). For others, not so much.

It's also possible that it's such an unlikely thing to hear that she actually misheard it and thought it said something nicer.

Am I the only one who would be delighted to be called Bitch (or any of the worst male-specific terms) by random professionals?

"Hey fucker, your prescription has been ready for pickup for three days. Be sure to get your lazy ass over here or else you’ll need to reorder it. Love you bye"

I would love this so much.

This is something I've been wanting ever since maps/driving apps came. I'd love to have Waze/GoogleMaps be angry when you miss an exit or miss the initial ETA by too much.

However, I don't think it fits the culture too well in the companies that could do it as trying hard not to offend anybody is of utmost importance.

Ya but language like this is how you know someone is good friends with you

If I'm used to waiting 2 days, and you get it down to 30 seconds you can call me what ever you want.

I'm more pissed if I'm waiting days for a response.

Me too. But I learned that not everyone is like me. And i general I also would not trust a LLM so much, that cannot divide between formal talk and ghetto slang. It will likely get other things wrong as well, humans will, too - so the error bar needs to be lower for me as a customer to be happier. I am not happy to get a fast, but wrong response and then fight for days to get an actual human to solve the mess.

I've grown up in various neighborhoods. In no context would calling someone a slur like that when you don't even know them be acceptable .

That said, it's obviously a technical glitch. Let's say it was something really important like medication, would you rather wait two or three days to find out when it gets here, or would you rather have a glitchy AI say some gibberish but then add it's coming tomorrow

I was assuming it was more of a text to speech error or typo in the records and it was supposed to say her first name there. Accidentally inserting super casual/offensive slang into a more formal conversation doesn't feel like a mistake LLMs tend to make very readily.

Fun fact, we fixed this issue by adding a #profanity tag and dropping the message to the next human agent.

Now our most prolific sales engineer could no longer run demos to potential clients. He had many embarrassing calls where the Ai would just not respond. His last name was Dick.

I find it odd that your engineer would make the system rely on instructions (“Do this. Never do that.”). This exposes your system to inconsistencies from the instruct tuning and future changes thereof by OpenAI or whoever. System prompts and instructions are maybe great for demos. But for a prod system where you have to cover all the bases I would never rely on such a thin layer of control.

(You can imagine the instruct layer to be like the skin on a peach. It’s tiny in influence compared to what’s inside. Even more so than, in humans, the cortex vs. the mammalian brain. Whoever tried to tell their kids not to touch the cookies while putting them in front of them and then leaving the room knows that relying on high level instructions is a bad idea.)

I wonder if the solution is to run the message through another LLM to make the message as polite as possible removing any profanities. Cost >2x as much to run though.

Maybe that was their first name, at least the one they put in lol

Reply on news.ycombinator.com