Customers don't want chat bots

It’s that you can’t pop some system atop a teetering ziggurat of shitty data from mutually incompatible systems and think the fundamental problems are going to just disappear.

Example: I am currently executor of my dad’s estate and trying to get various policies and pensions and whatnot transferred into my mother’s name. One company seemed especially clueless when I emailed them so I called. Part of the voice prompt on the phone system was something like “if the statement we send you comes from city A, press 1, if it comes from city B press 2”. Well that’s a weird red flag right there also a teeny problem: my statement comes from city C. I eventually speak to a human but try as they might they can’t find anything about my dad.

Eventually I write to the company at city C and they’re super helpful. Basically this company is formed from the corpses of various pension funds which just get welded together. There is no underlying integration at all between the systems. They are separate companies in all but name.

You could have an agi chatbot and its still going to be lobotomised by this level of disfunction. In my experience the data landscape at many enterprises is like this.

Which isn’t to say they can’t be helpful. I’d much rather chat with a bot than call a person, if the bot is capable of doing the things I need. The thing is nobody seems willing to let the system actually do anything except escalate to a person, so why bother?

It's extremely frustrating to have a real issue, take for instance a plane cancelled or delayed when travelling with kids, and not being able to talk to a real person who can actually help. Let alone having to wait 30-60 minutes to get a hold of someone.

Nowadays most large organisations are structured now that it's impossible to have a conversation with a human who actually has the power to make a decision. All they repeat is info that's found online and tell you to fill in a form or email.

In that case a chatbot is probably better for both sides, mainly for the person behind the phone who doesn't get verbally abused on a regular basis.

Sadly best way to get helped by corps nowadays is to tweet to them.

Chatbots almost by definition cannot deal with that, at least I've never experienced one ever. And I imagine that if they can understand what I want it's just too easy to code them to toe the company line and tell me to pound sand.

Empathy and understanding are not parameters, yet.

If you told it you wanted a refund for order xyz, and it immediately gave you a refund, I'm sure people would like it.

Fuck chat bots and the money-saving horse they rode in on.

A chatbot has no responsibility, it is just a useless parrot of the documentation. It doesn't serve any useful purpose except worsening the experience for the customer.

I can see the author probably hasn't done much customer service? I'd say maybe 1% of customer queries go in that direction.

I agree dealing with a bot in that situation sucks, but a big chunk of customer questions can probably be answered without human intervation by LLMs, assuming they have enough data about the organization and its products.

So I authenticated got through to the correct option on the first try, and it seemed like they were going to drop me into a queue to wait for a live operator. And then a recording informed me that "due to a high call volume, nobody is available to take your call." and dropped the call! Can you believe that. No option to leave a callback number or wait 4 hours in a queue, just terminate the connection. I called back and got the same thing.

The website does have a chatbot, but of course it is not capable of connecting to a human being and only capable of relaying the simple FAQs they've programmed into it.

I am pleased and impressed, for the most part, with the proliferation of Live Agent Chat for many services. I really prefer this over a telephone interaction (most of the time.) So it's good when I can easily break past the "virtual assistant" trying to FAQ me and get to a human.

But in this case, today, I suppose there were no humans to be found. Tragic. They have good compensation packages there.

But I really dig airline chatbots. I rebooked flights quite easily in the past with that. It gave me clear options, I could scroll the list of flights etc., way better than by phone. And at inconvenient times as well, so no hotline anyway.

That way I changed a very unpleasant being stranded for 2h in a hotel room in Geneva for a nice long stay in a five star hotel. Landing at 1, being in the hotel at 2 at night. Flight was scheduled for 6 something in the morning. This would've been gruelsome. So I rebooked my flight to Frankfurt for lunch and enjoyed the breakfast buffet.

Companies race towards having the shiniest chatbot or _virtual assistant_, without fully understanding all the implications and customer needs. Everyone wants to have their brand name associated with being "the first" or "the best" in their segment for providing such technology, no matter how trivial it may seem -- e.g. as the article points out: just wrapping already available info in a dialogue.

Given these incentives and high stakes, it's also hard to "exit" the bandwagon, due to various factors like reputation, costs, and the uncertainty around your competitor not doing the same.

[1]: https://danielmiessler.com/p/moloch-the-most-dangerous-idea

We didn't want phone menu systems, or to wait minutes, tens of minutes listening to muzak on hold, or to have to decrypt heavy foreign accents and painful latency on calls, etc either.

Customers will get what companies give them, which will be phrased as 'lower prices'.

I submitted an FCC complaint.

A few days later, a customer service rep called me. She told me that I could contact the chatbot. I told her I'd never submit an FCC complaint if I could talk to a real person, because I don't trust the chatbot.

> Customer service is one category where I expect a lot of these jobs, honestly, to just go away.

For long I have, and in normal conditions, I would have said hurray. Humanity invented a technology that automated some of its bothersome tasks, one thing less to do, we're collectively all for the better, freed to enjoy life or spend our time in a more productive way, whichever suits best.

But in this world, the fruit of this automation is always reaped by the same people, and they're not those whose job was automated. The latter are now in a shity situation, the former are now marginally richer and won't share with anyone else.

We’d need to have access to the specific company’s call categorization to know this. Just last week I was chatting with someone in their mid-thirties who was telling me how they don’t have an ATM card or access their bank through the app or website. Everything is done in person at a physical branch. They need money? They go to the teller. And I know many older people who are the same way.

This is why many companies already have automated phone systems that walk customers through the easy to solve problems with mostly rote formulas. Add in a chatbot that can be better at handling slight exceptions and now the company can solve customer issues without having to pay a human to do the interaction.

From my time answering phones for an internal IT department for a large-ish company, it’s clear that some people called just to interact with and talk with a human. Even if they could have solved their problem through an app before a human answered the phone.

> Instead, the reason people go to customer service is because of a question that’s so specific, or complicated, or gnarly in some respect, that there’s no way the app will have the answer: you need a human

I’d suspect most of these aren’t all that specific or gnarly, but just that it can’t be done through the customer facing app, yet. When switching from physical SIM to eSIM, Verizon’s website kept erroring and directing me to call. I looked online and many posts said it was quickly solved with a call. I called and they fixed it quickly. That could almost certainly have been handled in an automated way, just not yet. I don’t think they even asked any questions, other than confirmation style ones, which a chatbot could have done. I needed to waste time calling and waiting in a queue to talk to a human. The company needed to pay someone to be waiting to answer that call and spend a few minutes with me. Seems like a great spot for a chatbot.

My expectation from my limited experiences is that the number of calls that a sufficiently large business receives are not the specific or gnarly cases. I’d love to see real data.

Speaking with colleagues in private banking call centres (wealthy clients who you'd assume are more clued up). Many calls are basically comfort calls, holding their hand while they login into the bank account, they complete all the steps themselves with no input other than reassuring comments like "yes". Some probably see it as making sure the bank is liable if they "break something".

Every support call without exception I hear the excuse “due to higher than normal/expected call volume your wait time….”

A couple counter points that are kind of the same point.

1. I actually hate talking to customer support that doesn't speak my language as fluently as I do. It makes communicating the important part of what's wrong in these complex scenarios that much more frustrating.

2. Customer support, even when human, often is useless because their roles get restricted to scripts. Discover is a company that used to have amazing customer support, especially when dealing with the complexities of charge disputes. I listened to a woman refuse to give me any idea of what "evidence" I should submit to a charge disputes with a hotel that was billing me despite no services rendered. Instead she read off what seemed to be a script. I hung up, dialed back in, got someone new and they answered my question in great detail.

I think my point is the same as the authors; there should be a high bar for customer service but I'd take it one point higher and say we've been letting that bar slide for a while.

Last December I had to engage into long chat sessions with airline rep to enquiry about my flight change. I kid you not, they changed my first leg from morning to evening flight, and I only found out when printed boarding pass an evening before. Then I had to change my flight back due to work requirements.

It was not cheap flight: four legs with two connections from Europe to SE Asia business class, so I believe I received a good service. It took about three hours total. My website auth expired, I had to re-login several times. Fortunately, the dialogue kept going during re-logins. I had to wait a lot while call center employee checked everything, and once I found a mistake right before paying a new fare. I got transcripts to email. I would rate a tech behind this conversation as above avarage, but still the overall experience was painful.

If some LLM chat agend could do the same, but quickier and with less errors - I would just be happy about it. I don't see it as an impossible feat, most of the responses are already scripted anyway.

This answers the question of the article. What customers want is offloading responsibility without paying for it. There are many concierge level services at the premium end. Chatbots are an attempt at offering this type of luxury to the masses. And it may succeed.

The bot can handle basic functions like information, change, and cancel but on more complex servicing, it brings in a human agent. The customer always has the option to request a human agent. This lets customers self service for the simple stuff but still get a helping hand when they need it.

In surveys our customers seem to like this approach with some indicating that they like being able to use something like this on their own rather than having to wait for an agent to be available. I think, as long as you give your customers options a structured first layer can be helpful as long as you give them a way out to more hands-on support.

There just wasn't anyone I could contact. I searched for days. I filled out and submitted multiple large support forms, none of which I ever received an answer to. Eventually I contacted a local Norwegian organization named Slett Meg (Delete me) and asked them to either get my account back or have it deleted. They had it back to me in like two days.

But having worked technical support in the past I know that most customers don't do any sort of research at all and will (not happily exactly, but yeah) sit through two hours of phone que to ask the most basic of questions that is in the manual, on the website or anywhere on the internet really.

And there's a vocal minority that want to be able to buy a smaller truck in the United States, but they keep on building monstrosities with beds too short for a sheet of plywood.

Time to start realizing that Capitalism doesn't really offer consumer choice or optimizes for the best outcomes for people, it optimizes for the best outcome for the business.

Chat bots are annoying due to potential inconsistency, having to guess the types of things you ask for, due to their latency, but they've made major advancements recently, and we are only beginning to see what's possible.

If those companies using chat bots are failing to provide value, then they won't be competitive and they will go away. Why do we want to discourage them from exploring this new area? Experimentation is critical to understanding what will work and to the development of our society.

This is where the author misses the point. Chat bots will be te main interface in the future. Especially for these mundane middle of the road queries. Integrated into voice controlled ai buddies.

The author is spot on for the more complicated requests. Even though these will be triaged by a the ai buddy.

The problem with chatbots is that they make customer service time far cheaper than my time. If they have to pay $15/hr for somebody to argue with me about a $10 refund, they're motivated not to do it. If that cost goes down to $0.15/hr, however, you're incentivizing Kafka.

Only matters that require a transaction are to be dealt with with caution.

Ergo useless.

If I've arrived at your support form it already means I've done everything your articles say they do.

Yes the modem is plugged in.

Yes I've rebooted it.

Yes the cable is good.

Yes the house's electricity is on.

No you didn't report an outage.

files claim.

2 days later they replace a piece of cable 2 neighbors down and things work again.

...

The same was true for the previous explosion of AI. Autonomous driving. That was, and remains being, an almost entirely industry-side push to the marketplace.

The past of customer service is the complex phone system where you select a number from a menu, and they automate some tasks (like paying your bill, etc.) and sometimes you end up talking to an agent.

But the youngest generations hate talking on the phone and audio is a terrible medium for a menu. A chatbot can provide similar functionality, but you've got the ability to provide a much richer interface, and when you end up with an agent, they'll chat instead of talk on the phone.

As chatbots get better and better at performing actions, you'll have to talk to fewer agents, and waiting on hold will hopefully be a thing of the past

1. Properly defined and articulated customer service policies and procedures

2. A training program that adequately prepares agents to know, understand and apply those policies and procedures

3. Sufficient inherent capability and disposition of the customer service agents to apply that training

My take is that the (relevant) inherent capabilities of LLMs at even at this early stage are as strong or stronger than what I see in the human customer service agents I typically deal with. And the disposition of an LLM can be arbitrarily good through fine-tuning and prompt engineering.

As such, any company that gets 1. and 2. right has a high chance of providing a better support experience by replacing human agents with LLMs.

Answer: human.

Question: would you rather talk to a GPT-4 chatbot RIGHT NOW or a human after holding for 30 minutes

Answer: chatbot!

My regular contacts? Straight to human.

An unknown number? Suspicious chatbot verifying who they are.

Only matters that require a transaction to happen are to be dealt with a little caution.

Something more empowered and smart could be good I think

Frankly, such situations are pretty crap even with a human. Support systems are not set up to handle the high effort corner cases and usually you have to go through a couple rounds of escalation.

LLMs can (and will) do a lot more than providing chatbot-like interfaces. Instead, they will power agents that can manipulate a variety of tools and interface with extremely diverse systems, including end users, SMEs, etc.

> Instead, the reason people go to customer service is because of a question that’s so specific, or complicated, or gnarly in some respect, that there’s no way the app will have the answer: you need a human.

This is the thrust of the whole thing, and the problem it's not looking quite deep enough at what people want. People don't want a human - people want their problem solved.

A human is probably the best way to do that, but realistically, if you consider the role of a human in these situations, it's as an interface between the customer and an internal system. The human translates natural language into a series of actions in some computer program (or maybe multiple programs and also communication with other humans).

This is something that LLMs will inevitably be better than humans at. I just called my new health insurance company to get my member number. I spoke to a woman who first had to open up my account and spend some time looking at information. An LLM will do this instantly, and the aggregate time savings of this across all customer service calls will be enormous. That's on top of the fact that no one will wait on hold because all agents are busy.

She eventually realized she had to transfer me to another department. She gave me the direct phone number for that department then transferred me. Except instead of transferring me, she dropped the call. Then I called the direct number she gave me, and it was the department for cancer and complex medical claims. We've all had experiences like this.

Humans are not anywhere near perfect at this job. LLMs may not ever be perfect either, but they'll be better. A human at a call center either needs to be trained, which is costly and time consuming, or they need to be given rote instructions for all possible tasks. Neither of these make them particularly well-equipped for questions "so specific, or complicated, or gnarly."

Obviously this won't all happen overnight, but it's not tough to imagine that cases like Wendy's replacing their drive thru employees with chatbots can be successful in the immediate future. A Wendy's employee in that role just hears natural language and translates it to a series of button presses on a screen representing a very much finite number of possibilities. This is well within the capabilities of GPT-4. I think we'll see this more and more often for low-level customer service, with the chatbot passing the user off to a human for more complex things. The chatbot will move further and further up the stack of complexity until it's doing everything.

Hmm, what makes OP think a chatgpt can't answer this question? Given enough tokens passed to a ChatGPT session, a chatbot can eventually find out what's going on and provide a simple to understand explaination.