Hacker News Reader

Comment by janderson215 | Hacker News Reader

I see a lot of negativity in regards to using AI as a customer service agent. I have only spoken with 1 and that was calling Starlink customer support. It was easily better than 95% of customer support experiences I’ve had. My guess is the bad experiences have to do with bad execution. I’m sure some companies think they can just plug in AI and their job is done. Obviously that is wrong, but done right, the experience is far better than the situation we have today. I never have to repeat myself and if it’s tied in with your account specifically, it’s like getting escalated to a level 3 support rep immediately.

View on HN

The problem isn't that AI can't give good service. The problem is it's going to be intentionally limited in the service it gives and its ability to make decisions.

Every automated service I ever interacted with was built in a way that it would not help you with anything that incurs a cost to the company that built it.

I think frontier models are really good at solving technical problems. So if your support call is "how do I do this with the product" and there is a way to do that then I agree it can be a great experience. At the point where you need to involve the engineering team and maybe get bugs fixed I imagine the AI will push back because the entire reason the AI is there is to solve cost and if it ends up creating bugs or bugging the engineering team it'll start consuming money instead of saving it. It probably could be engineered to do this but the incentives are pretty strong not do that.

That is again an issue with company culture because the issues should be addressed anyway, AI or human.

Sure but: 1) most companies’ culture in this area is already bad; 2) AI tools enable and amplify the absolute worst of that culture.

> The problem is it's going to be intentionally limited in the service it gives and its ability to make decisions.

Apart from the Facebook AI customer support which would just blindly send MFA reset emails to wherever you asked.

Interestingly I had a pleasant experience with Vercel's AI support recently. I challenged a charge I shouldn't have had, it self-diagnosed to confirm that the invoice had been generated incorrectly, and actioned a refund! No human in the loop.

Yeah, I think AI support is great for those low effort calls where someone wants something simple and clear and company policy is set up to just give it to them. That’s why a lot of people like Amazon’s bot, it has direct access to account details and can auto-approve simple things like a return that is slightly outside the return window etc…

But what you shouldn’t do is try to dress up adversarial policies behind a friendly customer service bot and then fire your entire support staff. That is immediately obvious and will drive people bonkers.

It seems trivial to be upfront that its AI and for routine tasks only.

If a firm is so deceptive that they cant even do that, then they probably are deceptive in other ways too.

So its kind of a useful signal, ironically, for who not to do business with.

It's all about the execution. Most of our customer's prefer AI (per their feedback, so there is some selection bias) since they can get quick responses 24/7 rather than wait. That said there are still many cases where you need human intervention and it can be frustrating to have to get through the AI first (my experience with Stripe's AI support).

its surprising to HN tech crowd but a well implemented support agent gets higher reviews and successful resolutions than humans

Can you cite a source or study for your assertion?

personal info

maybe will be public soon with earnings calls? We’ll see!

Are frustrated hangups before any resolution is reached a part of that data?

yes, it counts as unresolved

c'mon listen to yourself

If it’s something easy enough for ai to handle then why isn’t it just an option on a site?

I think you are wildly underestimating how much of a hard on people have for calling businesses, even when there are better and faster options clearly available, like clicking a button online.

Staffing our phone lines is the absolute bane of my small (non tech) business.

Best AI I have used for support was Githubs, and even then Github has some nuance between its different enterprise offerings that it didnt quite grasp.

I find that 99%+ of support LLMs are just an extra hurdle that cant provide me with what I want.

I used Kogans chat LLM the other day, I gave it the details of my enquiry and it escalated me to a human. The human reviewed the case and then raised a ticket for a technical contact. I am not particularly angry at the LLM or the first human, but one of them is definitely redundant and wasting my time.

>My guess is the bad experiences have to do with bad execution.

This is the key, IMHO. I have not been able to access my Coinbase account for some time due to a login issue. (No error is displayed, but a successful login returns me to the login screen). The AI driven phone agent was actually very good at working through the problem, but there was no escalation to a human after that. When the agent couldn't solve my problem, it told me to mail a physical letter describing my issue to an address on 5th Ave in NYC.

    When the agent couldn't solve my problem, it
    told me to mail a physical letter describing
    my issue to an address on 5th Ave in NYC.

Honestly, this sounds like the hallucination of an LLM with no further options available in its prompting.

My last customer service call was fast (connection time) and friendly. The person wasn't sure so she said "let me ask my colleague $female-first-name" and I heard her typing. "Is $female-first-name an AI chat bot?" Yes, it was, and then she read out a whole answer, an excerpt that's already on the website. In this case the person added little extra value and I guess will be replaced soon.

There was an Intercom executive making a very interesting case for AI customer support on Patrick mcKenzie's "Complex Systems" a while ago.[1]

My main takeaway from that interview was that tier1 CS is a low-wage, high-turnover industry, so the amount of training agents can realistically get is limited. AI support is cheaper, smarter, and can take in more context faster. With AI, it makes sense to have a "support engineer" who identifies the edge cases and writes Markdown documents instructing agents on what to do in those cases. This actually lets you make agents much, much more helpful than human employees ever could be.

[1] (There's a high-quality transcript available) https://www.complexsystemspodcast.com/episodes/des-traynor/

Starlink's AI customer service agent confidently gave me incorrect information about shipping time estimates. It would have been better to have no answer at all than to have a broken promise.

I’m not affiliated with Starlink nor trying to defend them, but I would argue the shipping estimate comes from a black box and is totally unrelated to the AI agent. My shipping estimate in the app said expected to ship between June 22 and July 2, but it’s actually arriving tomorrow. I’m happy with it in my case, it I could understand if coming outside of the window (either early or late) is frustrating.

Exactly. Not Starlink, but Tesla, their stupid non AI allocation tracker said my model X would be ready in a month for about 13 straight months (Early Model X plaid allocation)

I was responsible for overseeing a Sierra AI implementation to replace our tier 1 support bot, which was not AI but a basic heuristic bot.

Containment from the bot alone shot up past 50%. Customer satisfaction with support went through the roof. And this was basic knowledge support with account awareness. We had not wired up any tools yet.

I’ve also really enjoyed Amazons support agent. Sometimes it can be really helpful.

Yikes, I find it hopeless, just takes me in circles.

If there is something that can be done without human approval, I don't want to call support to do it. I want to be able to do it myself on the company's website. I only ever want to call support if I want something that I think is going to be outside of the usual policy, if that makes sense.

So instead of companies working on their AI support execution, I wish they would just work on their website.

Can someone give me examples of the kind of support they want from a company that could be provided by AI, but could not be better provided by a well designed app and/or site?

Reply on news.ycombinator.com