Twilio is erroneously over-billing and suspending accounts | Hacker News Reader

Twilio is erroneously over-billing and suspending accounts

status.twilio.com

170 pointsamduser29119 comments

Rob from Twilio here. We are still actively working on this incident and aware that erroneous auto-recharge billing and account suspensions have occurred for a number of customers.

If this has happened to your account, please send an email to help@twilio.com where we have our support team working on making this right for everyone affected. We are spinning up additional resources to make sure every customer issue gets resolved as quickly as we can.

Link for this article goes to our status board which serves as the authoritative source of information on this incident. We will be delivering updates every half hour or as new information becomes available.

We are very sorry. This is far short of our commitment to you. We'll have more on the incident and what we will do to make this right later today.

Rob, best of luck to you and the Twilio team while fixing and recovering from this incident.

We've been with Twilio for years and have received excellent service. Billing issues are always serious, but its nothing compared to a call routing issue, and when it comes to call routing Twilio has been flawless. I'd much rather call my bank and discuss an overdraft fee then call a client and explain why their calls were not routed properly.

In the years we've been with Twilio, we have not experienced any significant downtime. Compare that to one of their main competitors that experienced eight consecutive hours of downtime during business hours earlier this year, and I'm confident we're in good hands.

Thank you very much for that.

We came up way short for you this morning - committed to making it right.

Seconding Willie's comments. I'm not your customer today, but seeing how y'all are handling this would make me feel confident to use Twilio should the need arise.

When you get some time (I'm sure you have none right now), read some of the Dreamhost discussion below. You folks appear to be doing an excellent job responding to this.

I completely agree. I've seen several public instances like this one over the past year here where Rob and the Twilio team have done what appears to be the Right Thing (from a customer service perspective). It's inspiring to see them do this, and to see that it is indeed possible to execute very well with a focus on customer happiness and "making things right".

Agreed, its a tough situation that Twilio is genuinely apologetic about. It happens but I'm sure it will be fixed.

Lot of work still to do to make this right, but thank you very much.

Peter Gibbons: You know, corporate accounting is sure as hell gonna notice $305,326.13, Michael!

I'm not a customer, and barely know what Twilio even is, but I'd like to say that this is a great example of an apology done right. Short, simple, to the point, describing in basic terms what you're doing, and acknowledging that you screwed up. So simple, yet so rare to see.

For people commenting about overdraft charges, Dreamhost went through this a while back. I was in college at the time, and didn't realize they'd drafted a second charge to my account. A series of small transactions later, and I was $300 in the red with very little income to fix it. They offered to reimburse overdraft fees for those affected, but I never was reimbursed.

This is also great as a "how not to tell customers that you just double billed them." At least Twilio is getting this right.

http://dreamhost.com/dreamscape/2008/01/15/um-whoops/

Step 1: Don't complain to the customer how inconvenient your massive mistake was for you.

What's wrong with the Dreamhost post? He specifically says: >"The end to this story is that of course, I’m very very sorry, we’re very very sorry, and I’m sure you’re very very sorry this happened. I really am. I understand the sort of problems that an unexpected large charge to your credit card (or worse yet, your debit card) can cause. If the tone of this blog post seemed a little light, I apologize I don’t mean to offend and I realize how serious an issue this is."

I can see why you'd still be annoyed about not getting your charges refunded though. What happened there? Frankly, if your bank didn't refund them when you told them what happened, they seem pretty scummy. I hope you dumped them as soon as you could!

The bank was SunTrust. They were very scummy.

As for what's wrong with the post, a ton. This was e-mailed to every single customer. For many people, myself included, this was their first notice that something was wrong with their finances. And it was the author's fault. The line you quoted was at the end of a very long post. The apology should have been right in the first paragraph. Instead in the first two lines he's complaining about how bad this is for him.

Then he makes a bunch of jokes. Then he explains why it was just tiny mistake. Don't feel bad if I missed the problem? I'm not the one being trusted with millions of dollars of your customer's money... [Edit: to me that bit reads as "See? You missed it too! Now why should I feel bad?"]

After all this, finally, he apologizes.

For me, this guy's mistake meant that I now had to worry about how I was going to pay bills/rent, and how I was going to eat. My situation was probably quite unique, but that didn't make it any less impactful for me.

To me the whole thing reads as "I just accidentally removed a few million dollars from my customers banks/credit lines - so now I'm gonna have a laugh with them!"

I can say with great confidence that I'm not immune to the mistakes that caused this to happen. But what I can say is that were I in his shoes, I'd lead with an apology, be very succinct about what happened, and finally list details as to how I'm going to make it better.

Edit: To his credit, if you read the next post he did a much, much better job there.

Edit 2: And also for what it's worth - I still use Dreamhost. Aside from this SNAFU they've ran an excellent service, and (again, except in this case) their customer service has been excellent in responding to any issues I've had. However now I always pay with credit rather than debit, and I don't store my card info.

Fair enough. It's probably a bad first contact on the issue. I read it as him fully accepting responsibility for the error and explaining exactly what had happened.

But then, it's easy enough for me to view it in a generous light, when it had no personal effect on me. I would no doubt have felt very different had it been my account that was wiped out!

Oh, and yeah banks suck, especially when you're poor. Been there...

I was about to say this too. There was a lot of vitriol on the comments as well, but all of them follow the frustration of being overdrawn. But I agree, if the bank or card company doesn't take this as a genuine error and erase the record, then it's a pretty lousy institution.

Maybe he could have worded it differently and shown less irreverence, but that may be a byproduct of the stress as well. It's a pretty shocking thing too and they're human after all.

My current web host uses them as a registrar for some years and they've been pretty happy with DH services.

See my other comment for thoughts on the e-mail/post (sibling to yours).

On the banks... "Overdraft protection" is probably the single most disingenuous thing the banks have done to low-income individuals. It's such a "squeeze 'em when they're down" tactic. I had all kinds of financial problems in college, mostly my own doing, but they were sooo complicated by this... service. Suddenly because of one mistake, 10 small transactions turns into an additional $320 expense. Three days later you get 10 little envelopes in the mail. And then the dread. "Shit, I did it again. Now what am I gonna do?"

To the best of my knowledge, SunTrust wouldn't let you opt out prior to the CARD act. Or, if they did, none of their customer service reps or bank managers notified me of the possibility.

Thankfully I manage my finances better now. And thankfully the CARD act makes overdraft protection opt-in (although for a lot of banks this is just another clause in the phonebook-sized account agreement you sign), and there are now a few checking accounts (ING Direct, Ally bank) that treat payment of transactions that cause overdraft like what it is - a line of credit.

My local college credit union has an optional "Overdraft Protection Loan" service. You apply for an $X loan in the usual way (so bad credit, etc. can still hurt you here), but if/when you get it, they don't give you $X cash. Instead, they attach it to your checking account. If you go over, the bank automatically moves money from the loan to your checking account in $100 increments to cover the overdraft. There's no per-use fees, and as long as you pay it back within a month, there's no interest charged. (Or very little, I'm not clear--sometimes there's an extra 40 cents or so on there.)

It's really, really handy and they don't advertise it at all. I found out about it when I accidentally ran up a dozen overdraft fees and talked to an account manager about how to keep it from happening again. I'd recommend that everyone see if their bank offers something similar.

That was a pretty terrible post. Very self centered. Just lots of me me/I's I's in the first few paragraphs I read. Pretty tone deaf to the real issue which would be customers like the above being impacted.

I'd say, as a general rule, don't put engineers into a PR role and expect it to go smoothly.

I think this very much depends on the empathy of the developers. We've a pretty strict rule that PR are not allowed to comment on technology; each time they have they've gotten it wrong and we end up looking "stupid". Now we have devs and PR work closely to get succinct and technically accurate updates out to our community.

This is, interestingly, a good reason to use a credit card (and pay it off in full every month) rather than a debit card. With a credit card, you at least have a layer of insulation between your bank account and an erroneous charge.

Indeed. I have a much better system in place now.

He who has the money has the power.

With a debit card, an erroneous charge means the other guy has the money, and you have no power.

With a credit card, you still have your money until you pay your bill, so up to that point, you still have the power.

This sort of thing has happened before to another company, but that was a case of human error compounded by the fact that there were no software safeguards in place.

http://dreamhost.com/dreamscape/2008/01/15/um-whoops/

At the very least, it would be wise for any company to implement billing features that will under bill rather than overbill. Under billing at least gives you the option of notifying users later (or you just eat it). Overbilling can sour a professional relationship to the point of ending it.

Unfortunately when automation goes wrong, it usually tends to do too much - double billing, aggressively canceling things, etc.

Having built several automation systems that bill millions of dollars, I can say that it's quite challenging to build things that have no visible output, run by themselves and often sit there for years without being touched, while everything changes around them!

It's not sexy work, but when it goes wrong the shit really can hit the fan!

Here's a choice piece of code we have in production along the lines of what you suggest:

    if prorated_amount > (cost_per_month * 2)
      raise "critical! we were about to prorate-charge someone #{prorated_amount}"
    end

It's never been triggered, but this is probably the only place in our system where we're actually responsible for an exact amount that ends up on an invoice, so having a reasonable upper limit seems like a no-brainer.

What if this code is called multiple times in quick succession? Based on the fact that I got a dozen separate emails over the span of 10 minutes, I'm guessing Twilio's overcharging happened in a series of transactions, not all at once.

This code -- while good for catching bad single calculations -- wouldn't catch that.

Well I don't think he was saying "cut and paste this exact code into your codebase" but rather that it is possible to write safeguards such as this... in this case, perhaps checking whether an account has already been billed this month, or whether there is already a transaction pending, or...

We can't prevent every mistake but we can make sure the same one doesn't happen again.

shouldn't it be >=, to catch double-charging?

We've been charged about 10k and going up...

Looks like their auto-recharge balance test keeps returning 0 or less than 0, it charges your card, and loop.

EDIT: Looks like they're doing something - the UI at least now displays $500, rather than $10k before. The only real confidence I'll get is when I can see todays account transactions from the bank.

I had like $240 billed, when there was nothing used... fuck.

I think that reaction is entirely appropriate in this case.

We just unsuspended the accounts affected by this incident and are currently working on the balances.

If you haven't already, please send an email to help@twilio.com indicating you are affected by the incident so we can communicate directly as we correct these erroneous charges.

We are actively working on the incident and have identified the accounts affected.

There will be account balance changes in the Dashboard UI as we address the erroneous billing.

If you haven't already, please send us an email to help@twilio.com so we can keep you directly updated on our recovery work.

Dear god I hope the $500 in shows in my UI isn't just a ruse to stop me worrying before knowing it's actually in the XXXX/X price range!

The account balance in your Dashboard UI will have some changes as we continue our recovery work.

If you haven't already, please send an email to help@twilio.com so we can let you know directly the resolution for this billing error.

Apologies for nit-picking at punctuation, but occasionally it matters :-)

"Twilio is over-billing ..." = Twilio is charging more than they should.

"Twillio is over billing ..." = Twilio has given up on billing. It's so over.

Just my personal bugbear - but sometimes it's worth getting this sort of thing correct. Apologies if this is just a US/UK difference, also (I'm in the UK).

I'm from the US and I also originally thought the headline was saying something along the lines of "Trilio [has ended the practice of] billing and suspending accounts". When I clicked on the article I realized that they were "over-billing" instead of "over billing".

Since we're already nitpicking, "overbilling" (without the hyphen) is probably better.

https://www.google.com/search?q="overbilling" vs. https://www.google.com/search?q="over-billing"

Not a single person was confused by this, including me, a British dude sitting in the middle of London.

Now you got me confused. You are saying that no one was confused, not even you. Or you say that it was confusing not only for a single person, but for you too?

I'm happy to take the karma hit I'll get for this, but IMO this is emblematic of a large problem with HN, i.e. asinine bike-shedding. I address an unhelpful nitpicking comment and get a nitpicking reply. It's turtles all the way down.

The fact that two of your sibling comments are from people who were confused by this is just hilarious.

My apologies, I clearly underestimated how many pedants comment on HN.

agreed. I was excited to see how they were getting around that pesky issue of billing people.

same here. I was hoping the story introduced a new paradigm of charging that eliminated 'suspending' accounts all together, such as pre-paid.

I see a lot negative comments here. Frankly, I understand people's frustration, but I think Twilio are dealing with this in the right way.

I was one of the customers affected. Luckily for me, my credit card automatically blocked the transaction after the 10th time I was auto-recharged, meaning I had a cap on how much I paid. I can totally understand the frustration of people discovering that they suddenly have 100's or even 1000's of dollars worth of charges to their credit card.

On the other hand, by the time I noticed that something strange was going on, Twilio's status page already said they were looking into it. I contacted their support right away, and they answered within 20 minutes, during what must be a customer support crisis.

And now RobSpectre is on HN giving very helpful answers.

So, no, it's not their finest work (as RobSpectre has said), but at least they're dealing with it quickly, and hopefully professionally as well. Let's let this play out before we jump up and down on a great company that we all love.

Thank you very much for the support.

Lot of work to do to make up for this shortcoming.

This is bad. It'll badly affect people with low bank balances and high overdraft charges - it'll be interesting to see how Twilio reimburses people for charges banks levy on customers.

Or if it's on a credit card and you hit your credit limit and they keep trying to charge. How do you reimburse for a drop in credit rating?

This is unlikely to happen given most cards report monthly with a 30-day delay. Assuming you're able to contest the charges there would be no effect on your rating.

Debit/bank users are definitely the most susceptible.

Additionally as far as I know there is no credit rating penalty for repeated attempted charges above your limit. That sort of information isn't even on any report I've ever seen. The increased balance may impact your credit score, but as you've noted it's unlikely this would make it onto your credit report unless Twilio is rather slow to resolve the issue.

However, running close to your credit limit does impact your credit score, at least according to everything I've seen on the subject.

Normally, if these errors occur and are dealt intraday, there should not be a problem.

So I would wait and see, how this is handled. I see updates about every half hour, so I suspect, that they will handle the rest as transparent as well...

Disclaimer: Not a customer.

I agree - this is not our best work.

We'll have more on what caused this incident and what we're going to do to make it right later today.

OP link directs to the board with our up-to-the-minute status on resolving this incident.

They should be ok if they reverse the charges before the transaction is completely processed.

If they void all these transactions you should be fine right?

I'm glad they (and most of us) are not in the business of making insulin pumps or other software where such bugs could have potentially lethal consequences.

You mean you don't want to see a medical supply startup that "moves fast and breaks stuff"? I'm surprised we haven't yet...

Disrupt Health!

With death.

Like Soylent?

Soylent hasn't broken anything... yet.

Don't medical people get around that by ensuring the first customers are mice?

Well, don't rule bad software glitches in medical devices out so fast. As a matter of fact: they can kill:

http://en.wikipedia.org/wiki/Therac-25

Yes the machine producing insulin pumps would be in a state of continuous integration/deployment. Then one day at 3am a developer on a binge of Monster Energy and pizza would push an unchecked bug to git and have it deployed.

ah, so that's why my insulin pump looks and acts like a pager from the 90s? I suppose that makes sense. Better an ugly pager than an accidental 10x bolus.

You should check out some of the newer pumps in the past few years - specifically the Tandem t-slim and the OmniPod. A lot more functionality, and a lot less pager like.

Perhaps with nano-tech we can eliminate this? https://en.wikipedia.org/wiki/List_of_withdrawn_drugs

Unfortunately you use very different software development practices which are designed to move slower for things where lives are on the line. Things more akin to Spiral and Waterfall.

I'm not 100% sure that "unfortunately" is how I would describe an approach that is explicitly deliberative and slow-moving, when for instance devices are implanted into people's bodies.

I've been using Twilio for over 3 years now and haven't experienced any significant downtime or service issues. Their support has always been amazing. Obviously this situation sucks but if there's any company I'd trust to make it right it's Twilio.

From a developer point of view, we all have bugs. I remember the first time one of my bugs impacted customers and it was awful. If there's any community that should be understanding that things like this happen in the tech world, it should be us.

Thank you very much for that shakes. Grateful we have you in our community.

We fell short of your expectations today. More later today on how we'll get this right.

I think the fact of the matter is as a developer I want to get angry at them and then remember this is my own worst nightmare - mistakes happen, the true test will be in how Twilio reimburse people.

Yeah, I see a lot of (completely fair) angry comments in this thread. But honestly the response has been pretty damn good for what happened, and we will see how it plays out. Any one of us could have made this mistake and Twilio is a great service, otherwise you wouldnt have seen many people here complaining at all!

Response is not good until everyone affected has their money returned to them and we explain what happened.

Very much appreciate your support through a difficult night and morning. More to come.

I work in telecom, in a slightly different vein from Twilio's and I deeply empathize with the pain of billing.

Even if Twilio's somehow able to ignore the pain of telco engine rating and least-cost-routing, there are still so many places you can make an error and cause something like this.

Kudos to Rob and team for owning the issue and I hope the Twilio folks get this sorted out (I'm confident they will). It's worth mentioning that I've never seen another billing issue of this size from Twilio so one in 3-4 years isn't too awful. Compared to AT&T it's probably generous! (Facetious but with a grain of truth).

Good luck guys and keep rocking. Here's hoping everything works out.

I'd be really interested to know what ends up being the root cause. I know properly transactional databases ain't so popular in some parts of telecom, but it's pretty simple to insert a record of the charge transactionally. Then the only real failure path is the lack of distributed transactions to the credit card processor, but it's unlikely you're going to run into many repeated commit failures.

Our experience: We run BigCouch for Database and BrainTree for processing. We're using DTH Software for our rating engine at the moment.

I don't believe any of these technologies have any failsafes built-in related to credit card processing (or overcharging), and so anyone who uses these systems has to perform their own sanity checks.

It's extremely rare for billing to screw up in Telecom, because, essentially, that is the crux of all Telco business. Twilio is probably using a homegrown or Non-telecom billing system, but I don't have any insight into their operations. I would hazard a guess that it's homebrewed though as I can't imagine a processing company introducing an error like this (stranger things have happened).

So yeah, I think this will be a one time incident, and frankly, their response was fantastic.

Rob from Twilio again. Our CEO Jeffiel just posted an update on the situation on our blog which can be found here: http://www.twilio.com/blog/2013/07/billing-incident-update.h...

Full text also here:

At 3:28am PDT/11:28am GMT, our monitoring systems reported an anomaly in our billing systems, which resulted in erroneous credit card charges and in some cases account suspensions. This incident affected 1.1% of customer accounts. The on call team immediately began an incident response, using the Twilio status dashboard at status.twilio.com to update customers at regular intervals. By 6:24am PDT/2:24pm GMT all suspended accounts had had service restored. The same problem briefly re-occurred at approximately 12:30pm PDT/8:30pm GMT, affecting 0.3% of customer accounts, which were immediately remedied.

At this time, the Twilio billing system is offline and account-balances are not being updated in real-time until we fully resolve the issue. We are actively processing credit card refunds, and you will see a transaction void or refund on your credit card statement shortly (most banks process these within 24-48 hours). While the billing issue is being resolved, all voice and messaging services continue to operate normally.

In addition to refunding erroneous credit card transactions, Twilio will also be crediting affected accounts an additional 10% of their last 30 days’ spend. We recognize that it’s not about the money, but our responsiveness to the situation that matters to you. If your account was affected, please consider this credit an acknowledgement of the inconvenience we’ve caused you. If affected customers incurred overdraft or over-limit fees due to this incident, we will also make them whole. We are in the process of contacting all affected customers via email. Additionally, we will be releasing a full postmortem on the incident once all events and root causes are known, as well as detailing the corrective steps we’ll be taking.

Our focus is on providing you the best quality service and experience, and we recognize that today’s disruption came up short of what you expect from Twilio. Please accept our apologies and know it is our mission to always be earning your trust and business.

As many have you have pointed out on Twitter, this is not a fun day for our team, and especially our engineers. We appreciate your support and your patience.

Sincerely, Jeff Lawson CEO & Co-founder

I suppose this is because it was written in haste, but you're an hour off on GMT times.

Haha, the one time in my life having an outdated credit card on file paid off.

As uptown said, providing valid credit card information creates an authorisation for a merchant to charge to you, if your card expires after that authorisation has already been created you can still be charged and that charge will still be approved by your credit card provider. This is a common problem for people who take out short term loans and then think they can cancel their card to escape being emptied out when due day comes.

Cancelling a card (or letting it expire) prevents new authorisations, it does not cancel previous authorisations.

Also there are automated credit card update services, so if you get a card reissued then vendors can get their card on record updated.

Better double-check. Sometimes credit card companies allow charges to continue to flow through old, expired, even cancelled / compromised credit cards. I learned first-hand that a card can still process charges after AMEX allowed about 7 months of Netflix charges to continue to be processed from a card number I'd reported compromised.

What may be happening there is that Netflix is connected to services provided by the card companies to stop interruptions to recurring billing for reasons like this. Long story short if Netflix receives an error code from the payment gateway that says something like "Card expired/Card changed" it'll make a call out to this service with the old card data. That'll do a look up and return the new data to Netflix who then updates their records and charges away happily.

My bank hit me with a fee when Dreamhost charged an expired card ... It was my fault for not keeping my account up-to-date but it does seem like the kind of thing their billing code could check.

Goodbye, twilio...

I'm very sorry to hear that, but understand your frustration.

As I indicated above, please shoot us a note to help@twilio.com so we can make this right for you.

Yeap had this ourselves. Auto recharge actually took $500 rather than $20!

Absolutely unacceptable.

As I indicated above Daniel, if you haven't shot a note to help@twilio.com indicating you're affected by the incident, please do so we can communicate directly when this $480 mistake is corrected.

This is one of the reasons I like to use my credit card over a debit card. Knowing that errors like this don't affect my mortgage payments makes me feel safer.

The web portal of my bank has a link ‘cancel’ next to every direct withdrawal which lets me cancel the charge immediately (at the cost of the people charging me, of course).

This has gotta suck. Many cards have steep overdraft fees.

My account was impacted by this as well, however, it looks like my credit card company blocked the fifth recurring bill (and I woke up to a call from the Fraud Team at my credit card), so no real damage has been done. Support answered my email within 10 minutes, and promised to help me clean everything up.

Very sorry for the inconvenience here and appreciate you opening a ticket for it.

More to come.

The worst part is that the calls aren't getting updated with prices right away... so if you built a system that relies on getting the price info at the end of the call, you probably aren't pretty happy now.

We are working cautiously on restoring full billing service to all accounts. We are trying to err on minimizing further billing impact to our customers as we restore service and, as such, taking each step carefully.

This will also affect the Usage API for some users. If you would like an immediate update when these are restored, use the OP link (http://status.twilio.com/services/account-portal) or send an email to help@twilio.com.

We're moving as expeditiously as safety allows.

Thanks, I have been in touch with my account manager... just making a note that developers should consider this type of failure to try and build resilient systems using Twilio.

Good to know.

Very sorry again.

Oh dear, not nice at all if you're on a budget

Definitely not nice for anybody.

We will make this right.

Hit -- but we only see an overcharge (3x). Service seems to be back up. Do you have a support number our ops team can contact (they're paranoid).

Very sorry for this mistake.

Can you have your ops team reach out via help@twilio.com and we'll get them connected with a resource to keep them informed?

Another example of the amateur nature of Twilio. Good for $30 a month developers but bad for businesses that depend on their phone system.

Mistakes like this really suck, but I have no doubt that Twilio will make it right.

painful reminder that you need to test payment processing code, by hand or however, after updating it.

i've run some ecommerce sites and this is basic stuff - new payment modules, time to test card processing works properly.

Should have used <= instead of <

Best of luck

Wow. Seriously? Learn to code.

Doesn't matter how good you think are. If you write code professionally for long enough, you eventually experience an event like this first hand. It's an absolutely horrible experience for a developer.

Being a professional developer is hard. That's why it's generally a high paying position, because you solve hard problems and are accountable for the results.

So stop making excuses. An organization of this scale should be taking serious precautions (against code errors, bad hardware, malicious attacks, network problems, compiler bugs, cosmic ray induced bit errors, etc.) when building a credit card processor.

If something truly extraordinary comes out to justify this problem, I may be sympathetic. But Occams razor suggests this was avoidable (through good process, not "I should have seen that off by one error").

I would think NASA has even more controls in place and still shit happens. All problems are avoidable in hindsight but it is inevitable that you will have problem at some point.

But robobrobro claimed they didn't know how to code, which is a completely different thing.

I work in banking and when a piece of our code went down due to failing to handle load correctly during a market announcement the business was able to say they lost, in that 7 minute period ~£90k.

How is that constructive at all? Shit happens. My code has bugs, your code has bugs. Just about everyone has something break at some point.

At least Twilio has some reasonable disclosure of these issues on their dashboard to inform people of what is going on. I'm sure they will rectify the issue as soon as they have everything back up and running.

Your code has bugs, my code has bugs, but when our organization fails to implement a system to effectively catch bugs - particularly catastrophic ones - it's a bit dismissive to throw a "shit happens" at it.

That is why companies should be made accountable for software errors, just like most industries.

How about we just stick to what we have (freedom of contract) and solve disputes according to the terms and conditions we agreed on, maybe with some PR gestures on top.

Imagine a world where Saas vendors would be liable for economic loss their software bugs generated. You'd have to fill in a huge questionnaire to sign up and everybody would be paying a tailored, very high price to cover all the risks. Every software vendor would be in the insurance business.

As someone once putted, "the software industry is like buying shoes that explode when you try them on".

We only get away with things like outsourcing, low quality work full of bugs, side jumpers into the industry, weekend developers, because no one is held accountable for software bugs and their consequences.

The only place people seem to care is when people life's are at risk, and even then we know how Therac-25 went out.

Well actually learn to write test cases and review code. Though even with that it's feasible to miss a bug.

Especially when the bug very feasibly might not be in your system.

In this case, Twilio is obviously calling out to a billing provider of some sort. I've run into situations myself where the billing provider returns a status message equivalent to "failure" when it actually was successful. In that case, many billing systems will try again thereby double-billing or worse.

Reply on news.ycombinator.com