They are already quite common and frustrating, but at least they realize it doesn't even understand the question half the time so there is a human escape hatch.
"Computer says no" is here.
Edit: so I'm not just negative and off topic, the article looks pretty good, kudos to the author. The engineering is cool, I just don't like the practical usage.
So you know what the fastest way is sometimes? Shooting off a support ticket so I don't have to be the one spending my time searching for the answer.
I don't know if there's a perfect solution to all of that but IMO there's certainly an issue if I have to google for information for the business site that I'm on. Chatbot, universal search, better UX, a source of truth, something... What I do know is 20 different subdomains with 20 different UXs and 20 different searches isn't good for the business or the user.
It depends on the size is the org in question, but at some point docs existing and being indexed is still not enough to find them. On the extreme side of that, AWS has docs for pretty much everything - yet I often fail to find the right page, simply because there's too much content and there's lots of pages talking about related things but not answering my question.
Sometimes it's missing docs, sometimes it's lack of searching, sometimes the most viable path to the answer is through support. (Amazon has TAMs for that purpose)
Totally agree and I feel for any any support staff, small or large, as it is unfair. I guess my point is insanely large, highly profitable organizations probably have a UX problem along with a much larger greed problem. No one should be stuck chatbotting or having non-existent support if they are a paying customer with Google.
And same goes for OSS libraries.
FWIW the announcement reads very boring to me but I guess I was expecting something else. Likely won't be super useful in a small-medium size company.
[1] https://stackoverflow.blog/2023/07/27/announcing-overflowai/
Even for things like support case generation for customers would be good... the customer interacts with the AI generating the ticket and gets the simple things like what you're running on and a more drilled down issue of the problem.
I get so many "I have problem, help" tickets with no information at all.
To be honest, I've had some good experience with some of them.
Amazon's comes to mind. I've been a customer for a long time, and I was shipped a faulty computer peripheral recently.
I briefly explained what the issue was to the chatbot and got an immediate response that a new order had been placed, that I should just keep what they originally sent me, there was no charge and it would be sent out priority.
And that was it. It arrived the next day and it worked fine.
Granted, it knew I was a long-time customer who has already spent a lot of money with them, but this was about as painless an experience as I can imagine. It sure beat clicking through multiple web pages of dialog options.
Google just had a net income for the quarter of $18B. Why do we accept the tepid to non-existent support of these companies? How much support does $1B cover?
'Cheap' support is typically terrible to the point of being worse than a chatbot, generally due to the terrible pay and conditions. As support engineers get good they generally move to higher paying jobs leaving a dead sea effect at the lower pay scales.
And when it's undercooked I want the confirmed(tm) human to soothe me.
“Sorry judge, my whole plead was nonsense and I quoted law articles that didn’t even exist, but that’s just because I used ChatGPT” — actual lawyer who wasn’t even disbarred.
Without human you just get a powerless regurgitation of FAQ and links that can't help in situations they didn't anticipate.
In support calls the confused user rarely has all the information they need to present to the person solving/finalizing the transaction and a bot can help reduce the human time needed.
We have been trialing this with our support team for awhile and a lot of higher execs were ready to sign off and dump rather large sums on some models, but eventually we were able to convince them to delay; the bots are just too prone to convincing but wrong answers, and our buy-in from clients is way too polarized: either they uncritically believe the bot or they are overly skeptical of the bot.
the tech is very cool, but I don't think that the technology or humans are at a place yet where it's ready for full on use outside of very controlled situations. I could see it being very useful as an addition to search fields or to maybe monitor the user's search inputs/actions and based on what the user is looking up, show some context-aware prompts.
What I'd really like to see is a bot that is extremely skeptical and shows the user its skepticism in an unambiguous manner; classify the data and make an internal flag where if the bot's skepticism is above a certain threshold, it finds knowledge holders it's aware of to work through the bot's skepticism and never act on the information until the bot has lowered the skepticism value after checking.
right now my experience with the bots I've played with is that they either just shut down the conversation without advancing it or giving the user paths for research forward, or the bot confidently just pumps out any answer it makes that fits as a response for the given query, and I think we need the bots to show skepticism and explain to the user what this skepticism means. (i.e., the user should be alerted that the bot isn't confident on an answer, why the bot isn't confident, alternatives that the bot understands to be equally relevant or worth consideration.
it can still be polite, but the bots need to share when they're out of their league and work to correct it; I think people will actually appreciate it, and the bots are well suited to this position because they have no emotional stake in the game, so users can get as upset as they want that they don't have immediate disagreement, the bot won't change its position just because the user is upset.