Hacker News Reader

Comment by datahack | Hacker News Reader

I think taking a slow path with AI around financial services is probably wise.

Unfortunately I don’t think other countries and multinationals will take the same approach.

So how do we avoid another arms race? This seems like a good public position, but ignoring AI isn’t the right private position if you care about your financial system.

View on HN

Nobody is ignoring AI.

There has been unprecedented budget spend allocated for AI in the enterprise.

The results so far have been poor where apart from customer service there hasn't been any game changing use cases to justify the hype. There just isn't that much you can do with an inauditable black box that is consistently wrong 5-10% of the time and depends on high quality input data.

And most enterprises have been doing ML/DS for years and so have already have tackled low hanging fruit using NLP etc.

Even for customer service I’m not aware of good use cases. But probably I’m ignorant.

Can anyone share examples? I’m mean well working proven ones, not marketing BS.

I would guess that by fine tuning an LLM on product manuals, installation guides, FAQs and vetted and customer support cases, one could create a competent support chat bot. Using RAG you could provide it with the output of your real-time status page for the product, and use a prompt that had the ability to forward issues to a human if the customer seemed unhappy.

I'm not sure if there's a concrete example of this in reality, but why wouldn't it work, greed and incompetence aside?

At work we use a chatbot trained on our docs, it’s pretty good and sees lots of usage. Of course people can also just read the docs, but many prefer the bot.

Same here, but adoptoion is a challenge. People tend to stop using it after the first instance of a bullshit answer - they lose trust completely. Which is, seemingly, inevitable.

Why is it inevitable? I think that the proper way to model it is as a multi-armed bandit - a bad response from an agent should reduce your likelihood of using it again but not to zero. If users have sufficiently good alternatives, I expect usage of the AI agent would drop down, but it seems to me that the other options are generally worse (and often similarly likely to give bullshit answers), such that users will over a long timeframe would settle on a relatively high likelihood of using the AI.

Github Support is an example: https://support.github.com

With information about how it was implemented: https://docs.github.com/en/support/learning-about-github-sup...

Anecdotally I’m using Kapi trained on our docs to help answer customer questions and its amazing at getting answers to level 1 support. We still need to edit the answers but editing something 90% right is much faster then searching the docs ourselves.

Quantum Street running on the IBM Watson platform. Manages $5bn a year in portfolios I believe.

That. Plus, with the AI act in the EU it will soon be illegal to do things like matching people to jobs fully automatically unless the toolkit offers top to bottom explainability and bias mitigation. In my industry there are tons of sexy demos - and nearly no productionized systems utilizing LLMs for anything else than content generation and summarization. And only where mistakes are tollerated. Certainly no gamechangers, though lots of enterprise-scale snakeoil salesmen.

Reply on news.ycombinator.com