Hacker News Reader

Comment by _se | Hacker News Reader

No one wants to talk to an AI for customer support.

i've gone back and forth on this over the last few months.

I started out thinking that we've all been conditioned by bad customer support chatbots whose only purpose is to look up facts from the FAQ and then tell you to call the real customer support line to actually handle your problem. the problem was that the chatbots weren't granted hee ability and authority to actually do things. wouldn't it be great if you could aks a bot to cancel your account or change your billing info and it would actually do it?

but then i realized... anything with a clearly defined process or workflow like that would be even better if it were just a form on an account settings page. why bother with a chatbot?

customer support lines run by humans exist for two reasons: - increase friction for things you don't want your user to do (like cancel their account without first hearing a bunch of sales pitches) - handle unanticipated problems that don't fit into the happy-path you've set up on the settings page

My worry is that business dudes will get excited about making chatbots that can do the former and they'll never trust an AI to be able to handle the later. So I'm now of the opinion that having AI customer support will only be used to make things worse.

Customer support isn't paid well, so they often aren't motivated to become very skilled beyond the level of a chatbot before they move on to other things. So the interface to bad docs doesn't matter much. And good docs are very hard to produce. AI magnifies problems when good docs are lacking.

> aren't motivated to become very skilled beyond the level of a chatbot

Everyone has some amount of common sense. The current state of the art does not, so it cannot make decisions. This is why these things can't currently replace real support beyond being a search function exceedingly capable of interpreting natural language queries and, optionally, rephrasing what the found document says to fit onto the query better

You can't even have these systems as first line support, verifiying whether the person has searched the docs because you can't trust it with a decision about whether the docs' solutions are exhausted and human escalation is needed. There currently simply needs to be a way to reach a human. I'm as happy as the next person to find that a no-queue computer system could solve my problem so I use it when my inquiry is a question and not a request, but a search function is all they are

Chatbots are loaded with issues. But I have also had a lot of issues with humans.

By the time I have an issue, I have usually covered basic ideas and FAQs already. Currently, I tend to use perplexity supported by ChatGPT before engaging online tech support, and I create a document for them before beginning.

> Customer support isn't paid well

"We choose not to pay customer support well".

I've worked at companies where customer support was both strongly supported, paid well and given the tools to do their jobs well.

They were incredible.

Microsoft used to have some managers do support calls. Absolutely amazing service.

There's a third case: dealing with folks who just aren't technically savvy enough to figure some things out on there own, no matter how intuitive, well documented, or fully featured your product is.

I think I'd rather troubleshoot with a well-scripted AI chatbot, than a human being who's forced into the role of an automaton - executing directly from a script. Just, FFS, let me escalate to an actual competently trained human being once I've been through the troubleshooting.

Actually, I do.

There's no wait in line. There's no waiting 2 min for each response in chat, or waiting 5 min on hold while the rep figures out what to do. And I've, shockingly, gotten issues resolved faster and better.

Using one semi-popular consumer app -- once it pointed me to docs on their site that Google wasn't finding because I didn't know what keywords to use. And twice it escalated me to send a message to the relevant team, where I got a response that addressed my problem -- and where escalation would have been necessary with a human call-center rep anyways.

The point is that it was far, far faster than any chat rep OR phone rep. And it's far faster to escalate too.

I'm sure this experience isn't universal, but I've been truly shocked at how it's turned what are otherwise 15-20 minute interactions into 3 minute interactions. At the same level of quality or better.

I've recently encountered one that just sends you in a loop, and there is literally no way to actually speak to a real person. Unless you want to give them more money; they're very responsive in that case.

This is a billion-dollar company you have definitely heard of.

Why don't you just name the company?

I'm guessing Amazon?

Definitely Amazon

Legal reasons.

(It's not Amazon.)

Then you get situations like this one where the AI invented a non existent policy (which the airline did not want to honor)

https://arstechnica.com/tech-policy/2024/02/air-canada-must-...

There's a non-zero chance that real humans working as customer service agents will invent facts, too (whether to try and be helpful about something they're not completely sure, or just to get a problematic customer to leave them alone)

Humans get things wrong, but are less likely to generate novel facts.

There's also no useful output whatsoever if you actually tried any troubleshooting yourself.

Never has a chatbot been of help to me.

I've had exactly one AI chatbot point me to the right documents. All the other interactions were exercises in frustration, and I've canceled more than one product due to shitty AI support. When I have a question, if an automated system could handle it, I wouldn't have a question.

I would if the AI chatbot could ever actually answer my question, but the conversation ALWAYS goes like this:

Bot: Welcome! Please tell me what you'd like help with.

Me: how do I X?

Bot: Please choose an option: [list of irrelevant things].

Me: None, I need to do X.

Bot: Okay, please try [instructions copy-pasted from the docs].

Me: I already read the documentation and it didn't answer my question, that's why I'm here.

Bot: I'm sorry, I don't know how to help.

Me: Can I speak to a human?

Bot: Welcome! Please tell me what you'd like help with.

Doesn't need to be AI, most customer support was already automated before ChatGPT rose to prominence. Hell, I developed a mobile website once for a power company that was basically a wizard / checklist of "Have you checked for known outages? Have you checked your breakers? Have you checked if your neighbours have issues too?" before they were shown the customer service number.

Human contact doesn't scale, or is prohibitively expensive. I sat with customer support a while ago (again energy sector, but different company) to observe, and every phone call was at least ten minutes, often 20-30, plus some aftercare in the form of logging the call or sending out a follow-up email.

They also did chat at the time, where a chatbot (which wasn't ChatGPT / AI based yet but they're working on it) would do the initial contact, give low-hanging fruit suggestions based on their input, and ask for their information like their address before connecting to a real human. The operator was only allowed to handle two chats at a time, and each chat session took about half an hour - with some ending because the person on the other side idled too long. I mean granted, the UI wasn't great and the customer service guy wasn't a fast typer, but even then it was time-consuming and did not scale. They had two dozen people clocked in, if they were all as fast as this one person, they can handle 50 support calls an hour at most.

It does not scale. This was for a company with about 2.5 million users who rarely need customer support. Compare with companies like Google or Facebook that have billion(s) of users. They automated and obfuscated their customer support ages ago.

24 people on 2.5 million users and you say it doesn't scale?

    2.5 million users : 24 support staff
    1 billion users   : 9600 support staff

If it scales linearly, that's about 10k support per billion users. I was going to say that a 10,000 person department for handling customer support sounds like it doesn't scale, but maybe I'm wrong, given that that is only about 5% of google's headcount.

Also in terms of costs: if those support staff cost 100 grand a year in salary and other costs, staffing the 2.5M-user company with those 24 support crew 24/7 (3 shifts, let's pretend it's equally busy at 3AM) results in some 25 cents per month per user that need to be priced into the product. The transaction fees on a monthly billing system are likely higher than that of a skilled support team if this is a representative scale for the industry

I frankly doubt the numbers, surely it costs more than this for an average company?

People want their support solved as quickly as possible. They don't want to talk to AI support bots because it's just an inefficient, error-prone wrapper over the documentation, which if you have an actual support need (as opposed to "I just haven't read any of the documentation") that kind of AI support isn't going to be helpful.

If you have an AI customer support that can actually support customer service requests and provide resolution, people will use it and be happy about it, or at least indifferent.

People who can understand what the AI is saying don't need the AI have problems the AI is too dumb and powerless to solve.

People who can't read the documentation aren't going to understand the AI's bad or even good summary of the documentation

This will depend on your product. I have a side project where I get a few support calls per day. 95% of the calls can be handled by just quoting documentation/FAQ verbatim. The customers are typically not very sophisticated computer users.

Broadly, I agree. And I am furious with Progressive insurance for requiring a smart phone/mobile app to file roadside assistance claims, and my inability to get someone real on a call.

But,

In this particular story, the people were asking questions that were answered in the instructions.

No one wants to waste their time answering stupid questions, particularly if they are a solo small shop who gets entitled people asking questions around the clock.

If it’s well-implemented, it’s fantastic.

This isn’t really customer support, but prisma (popular typescript ORM) has an AI that can answer just about any prisma-related question. It’s got a great RAG setup, can help think through novel scenarios, and always refers to specific docs pages and GitHub issues.

I think it’s made by a company called kapa. Those guys are gonna go far. That thing works SO well. I’ve been imagining how good life would be with a prisma-style AI docs assistant for things like massive, obtuse google APIs.

I want to talk to an AI for customer support as the first line so long as there is always a "Talk to a human" escape hatch.

And for less than about $50 a month, I understand why they need to spend less than half an hour per month to retain me. It'd be net negative profit otherwise. (unless they offshore, in which case the math is only slightly better).

And, yet, millions already do. The point of AI for customer support is to handle the very simple requests (maybe half). The rest, you can escalate. If AI doesn't know what to do, "Hmm, I'm not sure. Let me escalate your question/request to my manager." For most normies, this will work well.

No one wants to perform customer support either. Generally, people who are smart and capable of offering good support will stop doing it because there are more fruitful and enjoyable things for them to do.

I usually agree, but Lemonade (insurance) has an amazing support bot.

uh, I beg to differ. I felt like an autocomplete with a knowledge base and "direct links to the right email forms" would have been faster than the fake chat interface that the "bot" uses.

(Also, if you own a home in NY and use lemondade -- do know that they don't cover cast iron piping (extremely popular in NYC). I found that out at renewal...)

Cast iron water supply piping, cast iron gas supply piping, or cast iron drain/waste/vent?

the AI bot asked "cast iron plumbing" IIRC. Which in NYC -- Cast iron DWV is prominent, can't speak to the others.

I am implementing a support system for my side project, which combines the knowledge base (FAQ) with the chatbot. You can access all the answers by browsing the FAQs. If you want to contact me, you first talk to the chatbot, which has been prompted to only answer based on what the FAQ says. If it cannot answer based on that, it will make sure all the details and problem description is there, and then forward the ticket to me. In other words, chatbot is the first line of support.

Eh... I think there's a balance to be struck. You could leverage AI to handle the initial messages (90% of which are tire kickers or scammers) and funnel worthy exchanges to continue the conversation manually.

Once people notice AI is responding they will skip it and will request to talk to a human. AI will look the same as FAQs or Chatbots, people don't want to interact with them, they want a human being that is able to understand their problem exactly as it is.

Most chatbots are both useless and tedious to interact with. But I've also had plenty of interactions with human first-level support that's just following a script without any actual understanding. An AI would be able to provide a genuine improvement over that.

AI isn't an improvement for companies that already provide great customer support, but it has the ability to seriously raise the bar for companies that want to keep customer support costs low or that have a lot of trivial requests that they have to deal with cost-effectively

That is exactly what is happening at my employer, and it’s been really effective for trivial support, especially when it’s empowered to make meaningful changes on the customer’s behalf. It’s got large swaths of the whole UX in chat, with an authenticated session. You could see it being better a better experience than clicking around anyhow. It does a great job at search too. Lots of room to improve but it’s hitting its targets for reducing human support time and as a sales tool.

The right pattern is to put them directly in a queue to talk to a person, but have an system (AI or otherwise) in the queue to gather the minimal information. Like having the person explain the problem (and have something transcribe it) and have the system transfer them to the appropriate team after parsing their problem.

Or for really common cases (ie. turn it on and off, you're affected by an outage, etc), redirect them to an prerecorded message and then let them know that they are still in the queue and can wait for a person. 9/10 it'll solve everything, but also reduce friction of simple things that might be answered.

Maybe not.

But there are many ways in which AI can improve or help support. So even if "AI chat support" turns out to not work, AI can still be very helpful in automating support.

Like detecting duplicates, preparing standard answers, grouping similar requests, assigning messages to priorities and/or people and so on.

That's not what "AI" means now. "AI" now means LLM babble

Even LLMs can do many of what I mention. Categorizing, grouping, assigning prios etc. maybe not as good as dedicated AI trained for this purpose only (I guess many could be "simple" bayesian filters even) but good enough and readily available.

No one needs to know it's one ;)

Would you like to work with a business that treats your time and problems this way?

The only thing I care about is are my problems solved for minimal effort and time invested on my part. Whether it's AI or human doing the solving, I don't care.

Bots are designed to waste your time and save the company time:

- either it has no authority and is simply restating publicly available information

- it has authority but only after pressuring it. In which case I have to spend time to find the right combination of things to get what I want

If I’m calling it’s because I have a special need that is unlikely to be resolved by a flow chart.

Much better than an unengaged, unempowered exploited human.

Reply on news.ycombinator.com