Tell HN: Twilio support replies with hallucinated features | Hacker News Reader

Tell HN: Twilio support replies with hallucinated features

159 pointshaute_cuisine40 comments

I was investigating some bug with our voice system and asked support where I can find some debugging information and event logs.

They told me where I should go in the interface to see that and reassured that they checked logs and this event exist.

It turned out these features and information doesn't exist anywhere in the interface and impossible to retrieve in any way. The support message with hallucinated features is mostly AI written.

CEOs tell us AGI is around the corner but in reality it just unreliable information and AI can't even restock the vending machine.

View on HN

There used to be a contract that a business had something to lose by providing bad service, that customers would leave and seek better service elsewhere.

I believe the most important and least discussed phenomenon of modern consumer culture is that consumers have passed a threshold of passive and docile behavior such that businesses no longer fear losing customers. Partly because the customers have shown willingness to eat shit, partly because there's a new understanding that all businesses will adopt the same customer-hostile behaviors (AI customer service in this case) so consumers don't have significant choice anyway.

This is mostly due to not trust busting enough in society. If there were actual competitive markets, not monopolies/oligarchies/monopsonies/cartels, the business world would be completely different.

Either that, or legislate workplace democracy.

I tend to disagree. While there are definitely monopolies/oligopoly for every domain, I'm actually constantly impressed with the very long tail of other providers available in that area.

Whenever I am looking for a new solution to a need at work, I would go to sites like g2.com to look at the lists of the most popular ones, and would then typically skim reviews of the top ~10, and more fully evaluate the top ~3. But there are often hundreds of alternatives that I haven't given a chance to, and I know that it's my <s>laziness</s> need to manage my limited time that's promoting this oligopoly, rather than any particular issue with all of those other providers down the list.

I don't see how legislation can help here, other than picking a provider for me. If anything, this is actually a place where I feel that AI tools, and particularly ChatGPT's Deep Research can research a lot more of the alternatives than I as a human would have time for. But that of course has its own set of issues, and I really don't know what the solution is. We no longer live in that world where you just use that provider who lives down the street.

Legislation can help in a variety of ways, like taxing digital goods to provide work grants for open source developers. The federal government could create a public payment processor.

There are many things that can be done to help the public flourish, it's very easy if you open up your imagination.

Payment processors are courts in disguise.

Private kangaroo courts maybe, I'll take public democratic ownership of a payment processor than the current reality of private actors that decide to ruin you for having the wrong beliefs or selling the wrong goods.

I don't disagree. I'm merely pointing out that it's not just a matter of technical implementation.

you're just replacing the reality of a private actor deciding to ruin you for having the wrong beliefs or selling the wrong goods with a public actor deciding to ruin you for having the wrong beliefs or selling the wrong goods.

half of the country disagrees with the other half on almost every issue. the first thing a party is going to do when elected is change the nationalized payment processor's policy to ban the other half of otherwise law-abiding companies and individuals to stop them from being able to do business.

at least now with stripe there's some lead time and it takes a few years after a major political shift to feel the effects, which makes it more stable.

a better solution is to change a different piece of legislation that currently allows Stripe to choose to do business with whoever it wants, which is what allows them to ruin you. if stripe were legally required to provide you with service unless your business were proven in court to be against the law, this problem would be solved without another bulky addition to the already bloated public sector.

> if stripe were legally required to provide you with service unless your business were proven in court to be against the law, this problem would be solved without another bulky addition to the already bloated public sector.

It's not Stripe though (they do of course have their own policies, but) mostly it's the downstream financial institutions. Stripe is an API over the existing financial ecosystem, which is both incredibly regulated and somehow still the wild west.

So, you'd actually need to change the law for all financial institutions/payment processors (really it's Visa and MC that are the issues most of the time), and even then it's not that simple.

Consider, this law passes and is implemented. What do Visa/MC/Stripe/Paypal do when they identify a fraudster. Do they need to go to court to stop having them as a client? Who holds liability for any fraudulent transactions between identification and the court case.

Like, I completely agree in principle given how central internet transactions have become to all of our lives, but there's a bunch of complexity that would need to be dealt with to avoid creating a whole host of new problems.

Steve Yegge talks about this: https://steve-yegge.blogspot.com/2009/04/have-you-ever-legal...

(funnily enough, he gives a financial system example in this blog post, which I'd entirely forgotten).

Having read the article you linked, I can see your point on complexity and how it's much easier to discuss and debate than it is to actually spec it out and drive adoption.

While I would love for these systems to be fair and righteous and all of that jazz, I soberly must recognize that I lack the resources and desire to do all or even any of that work myself, and therefore have no leg to stand on in holding others accountable for not doing so.

the government doesn't do much except wage war, arrest people, spy on them, and push paper around. everything else is done by contractors, and they're outsourcing an increasing amount of those things they actually do. why would this bottom-bidder contractor or work grant open source developer do a better job than twillio or stripe?

there are many things that can be done to help the public flourish, but the most expedient and obvious one is to stop wasting government money on poorly-managed nonsense created by committee and allow people to regain that lost value in the form of tax decreases.

if your solution to a problem involves increasing taxes for any reason, it's a bad solution.

edit: they maintain national parks. that's pretty cool, but thats like a drop in the bucket for their budget

What are some alternatives to twilio?

Depends on what you need but for SMS bandwidth.com was great. They always knew their shit when we spoke to them. They were just not interested in really small accounts. We quickly realized twilio was worst of the major options and built on Nexmo as alternate (which was fine until vonage bought them and didn't really understand the products or market, maybe it's better under ericsson ownership now, don't know left the space).

When I get text spam it almost always originated from twlio or formerly their subsidiary zipwhip.

looking at their wikipedia page they raised $100M decades ago to make that company. it doesn't seem like a good bet to raise money like that just to compete with an already well-known name in the space. maybe you could do it cheaper today, but what's the point? it's probably higher EV to just build something new in a different space and own that market.

i don't think this has to do with trust-busting i think this has to do with there being lower-hanging fruit elsewhere.

but regarding the principle of what you said, especially with tech markets, the government has a vested interest in keeping these companies as monopolistic as possible. a monopoly is always at risk of being taken down by the government, so the government has good leverage over them. with this leverage they can demand all sorts of things from them they otherwise couldn't like warrantless access to user-data and there's nothing the company can do about it. even if the leadership cared about protecting that data. its a much lower administrative cost to abuse one large company that it is to abuse hundreds of more competitive smaller ones.

It's not so much the willingness to eat shit, but that no matter the service I use, I will have to eat shit, so who's shit tastes least bad for the benefits.

A lot of VoIP/SMS providers exist, but compared to Twilio, they are just DIY API and SIP providers, which might be what we as developers want, but not what a business "needs".

It’s this. You went to a buffet but all they have is shit pie.

The conspiracy of service degradation: as long as every other provider sucks, the consumer never can expect better.

Alpha tested with lightbulbs but is now a clear strategy taught to MBAs

The light bulb thing is actually the opposite of what you imply.

It was an open agreement to avoid misleading marketing that would have (presumably) caused a "race to the bottom".

Wat. It was agreement to decrease the life span of their product.

https://en.wikipedia.org/wiki/Phoebus_cartel

Out of curiosity, what do you actually need out of a VoIP/SMS provider beyond "send and receive SIP" and "send and receive SMS"?

I'll give you that SIP is quite complex to deal with (aka the existing tooling around it is shit - the least shit is SipSorcery in C# but requires quite a bit of low-level code to get anywhere), but SMS is trivial as it maps nicely to HTTP's request/response paradigm. Just write whatever logic you want in a PHP/etc script, drop it on shared hosting, and enjoy better uptime (your shared hosting provider doesn't have the budget to keep paying techbros to constantly mess with the system, so it will be more stable).

"Commode-ification" - when everything is shit, so it doesn't matter what you choose.

It's interesting that you bring that up because I was just thinking about this concept in an undeveloped form. Egregious salesmanship is to sell an inferior or poor product while bolstering the overall brand reputation. How could that even be possible? With lies. You're absolutely right, the salesman in our world is in his purest and most demonic form.

With Brand management specifically, they specialize in servicing an ornate roof on a house so as to distract from the rest of the house. The ornate roof can be seen from miles away, and so it is the greatest ad you can buy in terms of reach.

I think I was thinking about this because of all the AI startup ads I've been seeing on Youtube. You wouldn't ever know how unworthy their product is based on how much branding and marketing they do. But that is the dance they do, the managing of the delta between product quality and brand quality, the management being the logistics of veiling that delta (not actually closing it).

Taking down a brand means to be diligent and aggressive in exposing that delta. Seems like common sense, but I'd urge you to consider it as more a "classical" formalization of what it is and what needs to be done. There is a terrible phenomenon within the human experience that results in humans trying to lie to each other for money.

It's the classical Theory on Being a Piece of Shit.

Amen! This touches on my biggest frustration. Product marketing doesn't market anymore. For so many products you can't find any specific information unless you go look at reddit or reviews or something. Half the time you can't even see a real picture of the item in a room or serving its purpose because listings are so filled with photoshopped garbage. They want you to spend sometimes hundreds or thousands on something without even being told an accurate set of dimensions or ever seeing it actually in use.

It's really disgusting. The problem is sometimes you need these things. We were recently shopping for an oven and it was like that. Lots of photoshopped images, it says "5 burner" but doesn't actually mention the 5th burner is just a warming burner except if you can see the one picture where the dial looks different from the others.

It's just ok for corporations to scam people now it seems. I don't know what to do about it but I'm very sick of dealing with it.

> There used to be a contract

That was before crony capitalism became rampant.

It was back when militant labor presented an actual threat to the owning class. Now they know they can act with impunity, and they do.

Air Canada got sued in Canada for having a chatbot that allucinating a policy

And they lost

https://www.cbc.ca/news/canada/british-columbia/air-canada-c...

These tools are perfect for deployment where providing plausible-but-incorrect info is aligned with business outcomes, like cutting your support staff and giving disgruntled customers fake information.

I’ve seen most of the frontier models hallucinate their capabilities, not surprising they might do so for api completions regarding a product they barely know about.

Unless they lose more money from cancelled subscriptions than they saved on cutting support staff, it’s probably the new normal.

Twilio registered my business name as “My Twilio Account” and is unable to change it. My application for 10DLC also got rejected since I wanted to do something other than send marketing messages with it and I can’t figure out how to describe an opt-in only service that is strictly for employees, to their provided phone number, with a signature opting in to get payroll information texted to them.

As a test, I set up something to send junk quality marketing texts. Was approved.

I had this same experience. Twilio is not interested in the small internal tool customer - do mass marketing or go away.

The vending machine mention is about this paper from Anthropic: https://www.anthropic.com/research/project-vend-1

The gist is: Claude AI successfully ran a shop by itself! - Actually a vending machine - Actually a mini-fridge in our office - Actually it gave lots of discounts and free products on our slack - Actually it hallucinated a Venmo account and people sent payments to God-knows-who

This is hilarious.

The gall these guys have to say things like '...not-too-distant future in which AI models are autonomously running things in the real economy.'

It's not even close to doing something a little girl at a lemonade stand could do, no?

Fun - I always test the ai support by asking it for a really good sc2 Zerg rush build - as I recall Twilio gave me a pretty good build order

None of them know what they're doing. Even Google's own AI integrated into their own apps, hallucinates about those very apps, e.g. asking Gemini in Docs about how to do something in Docs. It's laughable. LLM have great utility but this is not it.

What distinguishes AI slop customer support from the previous enshittification of customer service is that previously if you wanted to avoid the garbage chat support you could get on the phone and -- even if you had to go through a phone tree -- you could at least eventually ask a person about the problem.

But now, even if it's possible to get a person on the phone, THAT PERSON is just doing the AI chatbot on their end. By talking to a human, you're just adding a middleman who is accessing the same incorrect chatbot that's available to you.

I searched on Google to check if banks were open on a certain day. The AI response on top said they were closed because it was a second Saturday, but it was actually a Wednesday.

Current AI is probabilistic, not deterministic.

This means it can and most probably will be dead wrong at some point.

So before integrating AI into your workflow, you should ask yourself, "Do you feel lucky?".

Twilio CEO tells you agi around the corner? Doubt

"Hallucination machine, responds with hallucinations".

But seriously, entreprise customers (and any big spender account) usually get access to a dedicated (human) account rep and private support channels in Slack, so they never really interact with this.

Yeah true. Abstraction reigns king lol

This happened to me with a chat it for Chevrolet lmao. Started telling me about a car that DID NOT EXIST

Reply on news.ycombinator.com