Comment by CharlieDigital | Hacker News Reader

Comment by CharlieDigital | Hacker News Reader

    > Anthropic is an AI company that builds one of the most capable AI assistants in the world. Their support system is a Fin AI chatbot that can’t actually help you.

This really cuts to the reality of AI hype: no, agents are not nearly as capable as OpenAI, Anthropic, etc. need you (or rather your C-suite, itching to fire you) to believe. They really, really need you to believe the hype. How can you tell? Cases like this and the fact that there are 5000 open bugs, constant regressions, ignored feature requests in the CC repo. The fact that Codex doesn't fully implement the simple and well-defined MCP spec for prompts. The fact that even CC has gaps with the MCP implementation...a spec that they created!

If the progenitors with functionally infinite tokens can't get this basic stuff right, everything else they are doing is just blowing smoke. I don't care if you can ship a kernel compiler or a janky "browser"; how about just make your software work? The smartest guys in this space, engineers making 7 figures in TC, with billions in capital, unlimited tokens, and access to the best models cannot make a simple customer support chatbot work.

But you! You're expected to deliver that customer support agent that's going to allow them to cut 500 people from payroll. You'll have it by Monday, right?

It's some Tai Lopez "Here in my garage" energy.

Let that sink in.

What if they built their company with poor support, so they don’t have to hold up to any standard ? But others companies have historically good reputation for good customer support, and maybe AI can help them automate easily 80% of easiest requests

Hear me out: what if a lot of the hype they are selling you is performative marketing that they absolutely need your C-suite to believe so they can cut more headcount? Then spend a bunch of time generating piles of code that is human unmaintainable because now you're using AI code reviewers, AI testers, AI QA. Then thrash around using more tokens when it invariably causes production issues and no one can read the code anymore except for their latest and greatest models with 1m context window.

Congrats. Thats the strategy of OAi and Anthropic.

Those are already automated by making your first question "Did you plug it in?", followed by "Did you actually plug it in?". Or industry equivalent. It's not like there wasn't any research into this in the past century.

Clearly they have sales and other teams as the important people within the company, with customer services being down the pecking order.

They don't need AI to automate their customer service requests, they just need decent forms with a standard issue helpdesk system. It takes some work to get right, but anyone with experience of building customer support services will be able to do that, to put most of the customer service team out of work!!!

The problem is that the Law of The Instrument applies:

It is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail.

So we have some AI 'hammer' going on here, and it is the wrong tool.

At a guess, 80% of the customer service requests are going to be billing related, with some need to provide refunds or free credits. Get the form right so it shows the right boxes and these 'easy wins' can show up as a big list that a customer service person has to glance over before hitting the 'refund everyone' button. You need the human there to take responsibility, plus they can work on the 20% of other tickets, once they have spent ten minutes clearing down the refunds/extra credits requests.

Google don't sell much to end customers, therefore no support. If I search Google for how to remove fonts from my computer that are not latin, and their AI bot gives me an answer that zaps my whole computer, I can't complain and ask for a refund because I never paid anything in the first place. Google do not need to speak to a single customer.

Meanwhile, Arsthropic have a commercial product with billing. They prefer not to do customer service, but they are stupid. Every contact with customers and friendly customer service is an opportunity to sell more to customers or to not have them hate you. This is why companies should do customer service, however, they also need to put CS at the heart of the org chart and acknowledge that a well run CS department raises revenue and is not a cost.

AI customer support is basically this: waste customer time by burning tokens instead of outsourcing to India.

It's really a bit fascinating. I've had Claude one-shot complex functionality... and I've had it be unable to debug its own .mcp.json file effectively.

Agents are very capable. Their implementation matters. I doubt many support agents have access to editing user records, so even if they can accept responsibility they won't be able to make any radical changes to your account to fix those. It's not AI problem per se, it's a product problem.

    > I doubt many support agents have access to editing user records

Why do you think that's the case?

Just because agents aren't immune to prompt injection doesn't make it so that they aren't fantastically capable

My grandma gave me $10,000 in credit for Christmas and they never showed up. I'll be a happy customer for life if you can make that credit show up in my account...

It only has a ~1 in 20,000 chance of working but at scale it'll go through!

The same reason why L1 support is generally useless (they also can't do pretty much anything other than escalate).

AI simply replaces L1 support: it's better at answering basic knowledge info and doing a RAG over the Q&A (plus cheaper and faster).

So it's just a coincidence that they can't edit user records? They can't get another agent to fix that, even?

Reply on news.ycombinator.com