Engineers' billing nightmares
github.com
github.com
Some of it comes down to picking the right tool for the job. The article doesn't touch on technology choice at all, but it matters. When I built a salary system, I deliberately chose a tech stack that had good numeric calculation support (so not JavaScript). These choices matter and have lasting impacts on your project. Same thing for handling dates, for example here .NET and Java have excellent standard library support for everything date related. Other languages, not so much.
Another point the article talks about is change: companies change how they bill over time; customers change their plans. Again there's nothing specific to billing systems here. All systems have to deal with change. All systems beyond a trivial size end up with migration scripts, and sometimes those scripts end up being productized and becoming buttons in dashboards.
I guess my main point here is don't underestimate what might seem to be an "ancillary" product when it's an essential service that makes sure cash keeps flowing. Treat every important project with the care and respect necessary. Allocate resources properly. Don't underestimate. Oh, and don't underestimate.
Something I've found is that there are two types of people when it comes to planning / forecasting:
SALES -- Wants to hear only positive things. Likes to ignore predictable issues, pretend unlikely risks will never happen, just go-go-go. Often sounds unreasonably optimistic to other people, which can be off-putting.
ENGINEERING -- Wants to hear all the bad things before committing. Likes to mull over all predictable issues to determine if they're show-stoppers and/or if workarounds are available. Often sounds unreasonably pessimistic to other people, which can be off-putting.
Both types can be in agreement that a project is a "good idea" while one type is listing only the upsides and the other type is listing only the downsides.
This can result in rather poor communication and missed opportunities.
Yeah, but this doesn't help, because the problem is that billing is perennially underestimated. Everyone thinks it looks easy, and offers "solutions" like those in this thread. And mostly they work, until one day you wake up and you no longer know how your bowl of spaghetti code works.
And then there is revenue assurance…
But yeah, also VAT calculations should be outsourced how ever possible. In the US, sales tax is at the postcode level so it gets really crazy.
It sucks you have to pay VAT even on digital goods and services nowadays. Ideally it'd be abolished entirely.
What would you suggest to outsource VAT calculations? We've yet to find a tool / platform we really like, that we could partner with.
Not sure how much it costs but doesn't look very SMB/startup-friendly...
I also have experience with wage tax calculation systems, and the solution we went with was browser automation that used a third party wage tax calculator. That stuff can be impossibly complicated -- and with many governments it changes every year too!
what do you propose as replacement? It's not as if the governments are going to let go part of their tax revenues
we had an account that was literally called "magic money" where these checks were deposited. Then when an insurance carrier shorted their invoice and we could find no rational reason why we would use the "magic money" account to cover it.
Javascript has one number type; float.
2) Wow, those downsides are insane. It's not constant time, thus can't be used in cryptography...? Has huge performance impacts and can't be casted to a "number"- how is this implemented? it can't be native INTs on the CPU?
Is this to optimise memory operations over a large plaintext/cipher text or is there some other advantage to it that is less obvious (e.g. more resistant to timing attacks?)
I use this approach to minimize the amount of time I have to waste on trivial money operations such as storage and api's, but this approach (number of cents) falls apart quickly the moment you need to do something as simple as calculate tax or a commission. So, still likely no getting around proper float math even if you may not really be all that interested in sub-cent units.
You don’t need to work in decimal cents. Many work in thousandths of a cent or smaller. But you want an infinite upper and lower bound. Even 64 bit integers only has so much upper range depending on how far you subdivide a penny. You also want the precision guarantees when you take those numbers and do more than just add or subtract them.
With all numerical calculations, you need to be careful about losing precision on mathematical operations, so it's a good idea to think about things like this even if you use a decimal number system with a very wide dynamic range - it won't be very precise all the time.
One (from memory, not exact) calculation I've had to implement for a customer: Multiply prices by whole numbers. Prices are given in cents subdivided to 5 digit precision, result should have 10cent precision, rounded down. Then, for the tax calculation, get taxes from some tables, 3 different taxes for each line item. Apply the first two taxes, Euros with 10 digit subdivisions, then round "according to trade custom" (which is the usual "up from .5 cents, down from below .5"). Then add the third tax, Cents with 5 digit subdivisions, round down. Then add all the line items and their taxes, separately as well as together.
And if there is a Skonto, take care to properly unravel the tax stuff again.
I've had to do it in plain Typescript, it is possible, but it really sucks. Those days when you yearn for COBOL... ;)
In the end, if you let a third-party provider run your billing you'll be dependent on their solution, and if something is missing that they don't care to provide you're just out of luck, or back to building it yourself, which is why these third-party solutions aren't terribly popular. It's very difficult to build a standardized solution that just works, so relying on outside software can make your problems worse.
> It's very difficult to build a standardized solution that just works, so relying on outside software can make your problems worse.
I want to see you doing billing without software.
Also, Stripe is not available for every country under the Sun because it's U.S. based. Not that Lago isn't, but at least Lago is open source so you just use it locally.
--
In fact, we outsourced part of the software for the late payments collection process to a reputable SaaS and the results were not great.
My take is that if your needs are fairly standard, you can benefit from such service. Especially your time to market will be shorter. But you won't face any disaster should you decide to do it yourself. If you want to innovate how you bill people (most Fintechs do), you should develop in house.
As a random example, Azure generally bills for resources on an hourly basis.
Suddenly, months, years, calendars, etc... cease to matter. Hours are hours. You can use UTC for everything, ignore time zones and daylight savings, and aggregate however you like, or how the customer likes.
Similarly, consumption-based plans are relatively straightforward. Simply multiply unit costs with the per-unit costs. You can even blend this into the hourly model above, where each hour has a "units consumed this hour" and a "unit price per hour". This way you can handle things like changes to the unit price fairly accurately.
With cut-overs between plans, the trick is to just bill for them as separate line items instead of one thing going through transformations. E.g.: 220 hours of Standard SKU and 500 hours of Premium SKU this month. It doesn't matter how many times the customer changed their mind this month that way.
A nice trick is that data like the above superficially sounds like it'll be huge in a database, but column-based compression works wonders on it. Any analytics database engine, columnar format, or reporting engine should reduce it by at least 30:1 or more. Also, you now have a ready-made reporting system that allows slicing and dicing! In some database engines like SQL Server, this is as simple as making the main billing table use a Clustered ColumnStore index. In more cloud-native land you could use Parquet files in S3 buckets, or whatever.
I'm not saying that complexity doesn't remain, it's just that complexity shouldn't be voluntary. Copy a working model that matches your industry, and then only the essential complexity will remain.
An example of a hard-to-solve essential complexity is currency conversions and taxation when dealing with international customers. That's an externally imposed set of problems that can't be hand-waved away.
Thank you for taking the time to share that.
This is a wonderful example of how engineers underestimate the complexity of billing.
> Suddenly, months, years, calendars, etc... cease to matter
No they don't. You don't send a bill every hour, or even every day. Typically you invoice monthly. And if you invoice monthly, what date do you aggregate the invoice? Setting aside the obvious problem that months are not homogenous, do you start on the anniversary date of signup, or the start of the month? Well, anniversary date is generally better, because it spreads out payments, collections and cash flow over the whole month, which reduces a lot of effort in collections and finance, and reduces the impact of any issues - but it's also significantly more difficult to engineer; how do you bill February if they started billing on the 31st of January? Monthly billing is far easier from an engineering perspective, but is not generally the ideal method from a business operations perspective. And get this, if you start with monthly billing and have significant income, and then you want to change to anniversary billing, you get all kinds of income recognition (P&L) problems that you have to deal with; it's really complex.
And what do we mean by "signup", anyway? Do we bill services individually based on when they are created, or do we bill based on the account creation date? What if there is an account hierarchy?
Hourly or unit based billing does literally nothing to solve these problems, because they are endemic to the billing domain.
> Similarly, consumption-based plans are relatively straightforward. Simply multiply unit costs with the per-unit costs.
Right - and what happens when you raise or lower your prices? That has to happen at a particular point in time, which is typically in the future. What if we give tiered discounts (first 1000 minutes at 2c, next 1000 minutes at 1c)? In both cases, we no longer can keep a single per-unit cost, but rather we end up with a cost matrix, which can become really complex when you add other factors (such as discounting).
Also, what happens when you are charging, say, 5c per megabyte... and your incoming CDR feed gives you bytes? Do you configure this as 0.00000005 cents per byte? When do you do the rounding? On an hourly basis? Daily? Monthly? What do you display as the unit price to your users? What if another plan is charging 5c per Gigabyte?
> An example of a hard-to-solve essential complexity is currency conversions and taxation when dealing with international customers. That's an externally imposed set of problems that can't be hand-waved away.
Currency stuff should be very easy in practice, but I agree that tax is hard. That's why you need a multi-stage pipeline for invoice generation. But that's perhaps a story for another post...
It's easy in theory: 'just do it', 'just copy', BUT the challenges are in - the tiny details (that can have a huge impact: we're talking 'billing' and 'money' here) - the maintenance: it's a living organism: you will inevitably change your pricing, your payment processor, the way you bill, your back-end/front-end etc
Technically the point of TFA is to market billing software. Billing systems are complex in that they can consist of many different moving parts, but billing system are also simple in that they are easily understood. Billing has to be easily understood, else you'll quickly scare your customers away. Complexity and simplicity are not necessarily at odds with each other. The meat of TFA is mostly about how billing systems can quickly become a time consuming endeavour which again isn't at odds with simplicity.
I disagree. Billing has to be correct - that’s billing rule #1. You can have an easily understood pricing plan, but implementing that plan correctly and consistently in a way that you can defend and audit can be very complicated.
Just take the simple example of implementing anniversary billing for a customer who signs up on the 30th of the month. What date do you bill them when the month has less than 30 days? What about when it has more than 30 days? What about when your customers are spread through 4 time zones? What about when daylight savings starts and ends? Do most engineers understand time and time zones well enough to get this right?
These questions are fundamental to correct billing at any kind of scale, and the answers are not simple, or at least I don’t think they are.
We established earlier that billing is likely to be complicated, but that doesn't mean it isn't simple. Simple things can be complex. Simple refers to ability to understand, while complexity refers to scale of interconnected parts. They are not at odds with each other.
> the answers are not simple, or at least I don’t think they are.
I think you can make a strong case that business isn't simple, but billing systems only need to serve the needs of business after business has already figured things out. You cannot bill at all, even without a formal system, if you haven't answered these questions. A billing system merely needs to adopt the answers that are already understood.
I’m sorry but I consider complexity and simplicity to be antonyms - and so does the thesaurus! https://www.thesaurus.com/browse/complex
> billing systems only need to serve the needs of business after business has already figured things out
Billing is a fundamental part of your product and your GTM. Getting it right gives you an advantage, especially in terms of trying different approaches. Getting it wrong can be a burden on the business, especially if you find out that you need to re-price.
As someone who also works in billing and subscription management, I 100% agree with everything you said and want to add an extra thing:
Even when the answers ARE simple, there's so many of them that you end up realising you want to pay someone else to make it their problem instead.
This is a dangerous strategy for a startup. A big company can afford to make a mistake and fix it even if it costs tens of millions. If you copy their mistake before anyone has realised it's a mistake you probably won't be able to afford to fix it.
It's also very likely you'll only be able to copy the externally parts of what the big company does, without really understanding why or what's going on behind the scenes. Really, you're advocating cargo culting a solution. That's a bad idea.
Take anniversary billing, it seems so simple, just store the billing day-of-month in the database and run the billing using the sql:
select do_billing(customer_id) from customer where billing_day=extract(date from current_date)
What almost all big companies do is pay someone to make the problem go away.
Out of curiosity, does your system use a NoSQL database?
> Right - and what happens when you raise or lower your prices?
The billing table has "CustomerID, ResourceID, HourID, Units, SKU_ID" columns. You raise or lower your prices on the hour boundaries. The new rows inserted after the price change will have the new SKU ID with the new costs.
Irrespective of when you send an invoice to the customer, the report simply does a "GROUP BY" over the (CustomerID, ResorceID, SKU_ID) columns and some range of the HourID column. You sum up the Units * SKU_ID.UnitCost value, or whatever.
This isn't exactly rocket science for any normal database engine.
I suspect the error people are making is not recording the change-of-billing-model as a "slowly changing dimension", and instead trying to do complicated rules-engines in code...
No, the last system I built was using PL/PGSQL. It's brilliant for this application.
> The billing table has "CustomerID, ResourceID, HourID, Units, SKU_ID" columns. You raise or lower your prices on the hour boundaries. The new rows inserted after the price change will have the new SKU ID with the new costs.
Like I said, this is a very simplistic way of viewing how to do billing. Sure, today you only have the hour dimension when choosing a price. But then sales asks you to give a discount to a specific, very large, customer. For only a single resource. For only the first three months. Suddenly, and very predictably, your billing is not so simple any more.
The problem is not that simple billing is difficult. The problem is that billing is not simple.
I'm not sure if I'm missng some part of the complexity here, but with the solution jiggawatts proposed, wouldn't the above scenario be fairly trivial to solve?
Just implement a new SKU for this special one-time (more like first time...) discount that the account manager applies to the account.
If you want to be fancy, you could code in (or leverage one of the dozen cloud tools, depending on where you're deployed) to enable the SKU for the customer at period X and remove it at period Y and replace with the previous SKU.
To ease reporting, in your main SKU table you could have a parent_sku column and a sku_type column explaining that this row is a special discount child SKU of the parent_sku. Add basic databse restrictions and validations.
Billing is complex and requires lots of domain knowledge, but IMO it's not really THAT hard from my experience.
EDIT:
IMO a harder scenario for any billing system to deal with is account manager A made a sweetheart deal to make customer B less angry, removing one SKU completely from last months bill, giving them a 25% discount for the next invoice, and giving them some freebie.
These types of requests usually end up with a lot of issues unless the business logic promised to the customer already exists.
You can also argue that we can just "get someone" to apply discounts manually, but isn't that the job of the billing system? Manual labour doesn't scale too well, and a lot of mistakes get made - asking a customer to return an accidentally applied discount is a very, very difficult conversation, and I've worked with companies who simply had to absorb years of incorrect discounts because doing so was the only way to keep the customer.
1/ Should it be a discount on future invoices, or a refund (an angry customer will ask for a real refund: aka $$ on their bank account)?
2/ Do you want to keep track of how much has been discounted/refunded to which customer (so that you can identify deeper trends: if a segment is being discounted many times: maybe you can adjust things, or maybe it's fraud?)? In that case, just erasing a line doesn't help
3/ If you refund, how do you deal with taxes that have been paid already?
Btw, we've written a quick primer on Refunds, Coupons, and Credit Notes: to lay out the pros/cons
https://github.com/getlago/lago/wiki/A-Primer-on-Refunds,-Co...
If memory serves, we ended up solving this by pro-rating the tax against the refund amount, on a per-item basis. And, I think, some smart accounting.
Oh and then you have the problem of when tax is becomes payable. Is it when the invoice is raised (accrual based)? or when it is paid (cash based)? And if memory serves, it depends on the size of your company. :head asplode:
hah yeah i've tried that a couple times over the years, it never worked.
Another aggravating thing i ran into is in the industry i was working in (pharmacy) prices are calculated using a contracted formula. The pharmacy (my system) calculates a price and then sends the rx to the insurance company. They calculate using the same formula (hopefully) and if their result matches or comes out greater than what we calculated then they pay the invoice. However, if their calculation came up less than what we sent they'd pay their amount. It was a cat and mouse game constantly.
Well one very large insurance carrier had a bug in their system that under the right conditions would come up a dollar short for a very common rx. I understood what was happening and called and called and bugged them to fix the issue. Finally, they just ended up saying "we're not going to change the system. end of story.". It blew my mind, I was right out of college and too naïve to consider they'd just flat out refuse to fix the bug.
My customer complained, but in this case they prevailed because the contract said nothing about rounding.
This can be really significant. They ended up having to compute the charges and send the carrier a bill for the difference. It was worth tens of thousands of dollars per month.
Love this quote!
> You raise or lower your prices on the hour boundaries. The new rows inserted after the price change will have the new SKU ID with the new costs.
In the big systems Im familiar with metering, pricing, and billing are discrete. Yes, usage will be metered to a SKU. But there are arbitrary pricing plans that denote price/tier/discounts per SKU. So the price is only joined to the SKU usage, per customer, at bill generation time.
As a software engineer with >25 years of experience, now running my own SaaS business and dealing with subscription billing, invoicing and tax issues:
1. I agree with everything you wrote. And more. Taxation introduces an entirely new level of complexity on top, and lawmakers design laws for humans, rarely thinking about automation. If you don't believe me, look into EU VAT rules, and enjoy the mess with GB/XI prefixes, the case of AX, and parts of ES that aren't really in the EU VAT zone, but look like they are.
2. The responses to your comment are typically dismissive and are to be expected from this forum. Usually the replies are from people who do not do the thing you do, but think that the thing you do is simple and have all kinds of opinions about it. I used to care more about replies, these days I just smile and move on.
2. I think it’s often young people who haven’t encountered externally-imposed, non-negotiable complexity yet. I try to take it easy on them because I suspect I might have been like that once :)
For example, people building foreign keys to prices and receipt items. In what cases do you expect to retroactively change what you charged all customers? Copy that data to the receipt item, don’t give your accountant a headache and heart attack when you change pricing. I’ve also seen people using those same kinds of rows to determine what features a user gets access to. Don’t do that, use capabilities/roles. When a payment succeeds, update the user’s capabilities. When a payment fails, remove capabilities. This also allows support to just give a customer capabilities without a line item or changing their plan, because some customers are just annoying and it is less expensive to give them a feature. It also allows more easily selling custom-tailored solutions to bigger clients since the features aren’t linked to line items.
This article is more about usage-based billing, which is probably harder to get right, but feature-based billing is something I’m more familiar with and it’s so annoying to see all the ways people will fuck it up.
I rewrote the entire system to snapshot the committed values at the time of purchase and fixing all the old invoices from database backups.
I can't think of any database that makes a row strictly immutable, as in carved onto the platter immutable. People will find ways to undo any soft-guards you put into place, all in the name of being helpful.
Feature-based billing is super hard too, and people underestimate the complexity even more.
We were genuinely surprised that some companies wanted to use Lago even if they ""only"" had a "feature-based/subscription-based" pricing, that someone new would consider as "trivial".
> Some of this complexity is self-imposed
Having spent years working on invoicing & payments systems, the business will impose complexity (which you can only say no to a certain amount of) and then legal will pay you a visit and explain all the many rules that apply to billing & invoicing that vary across each country. Internationalization isn't just a tax complexity.
Billing really is complicated, but the people who set the policies don't understand that. They want to be able to say "Hey, let's give people a 10% discount if they sign up for a yearly bill, but also don't have this service over here, and for governments and students let's cut that down by another 15%" and so on, basically "billing by English spec", and because they think this is trivial and easy it gets handed as a spec written in stone to the billing team. Even in engineering-type organizations, treating this as an engineering problem is often a foreign concept to the org, so the idea of having the eng team estimate effort and then go back to the original requestor and make sure they really want it 1000 engineer-hour's worth, and that the rest of the org is willing to dedicate that much time to it, is generally not done. (Or whether or not there's an alternative thing that could be done at 1/10th the price that achieves the original goals, e.g. "we don't have annual discounts built into the system but we have this other fairly-close concept already built in, can we just slightly tweak the ask to use that, rather than having to implement a whole new complicated concept?") Even a company that has learned the hard way that "Let's just change the button to blue and move it down an inch" is more complicated than it looks thinks that management can just write whatever billing policy they want and there are no costs associated with it.
On a more pure-engineering front, engineers have the tendency to want to implement billing discounts and such by lying to the usage database. "Ah, we will implement this 10% off policy by telling our record-keeping system that they used 10% less stuff." I've banged on how important it is to never lie to your database a few times, and the billing database is what really brought this home to me. This way lies madness, as you start to stack lies on lies on other lies in the database and then counter the lies with other heuristics and then counter the heuristics with other lies and before you know it your database contains no truth anymore, which is a problem because neither does anything else. The database was supposed to be the source of truth.
The very first startup I joined in the late '90s tackled this problem. The company tackled billing and all of its complexities by building a SaaS product before SaaS was a widely used term (even the company didn't call it SaaS for a few years). The level of complexity was very high. Creating rules, calculating charges, dealing with taxes, ensuring things were auditable, and making it all performant were all huge challenges.
Basic billing of hours x rate is easy, but (as you've noted) it's a rare person that is content with that. I cringe every time someone says "We'll build it ourselves" when it comes to billing. Even for simple things like the 10% discount (new customers only, plans > $9.99/month, discount expires after 3 months) get complicated very quickly.
Unfortunately, it’s really easy to assume everything is simple until the system gets complicated enough nobody understands it.
As noted, trying to fit billing into our stupid calendar system is tricky, because the calendar is stupid. Then, if you want plan upgrades/downgrades, you need to do pro-rata adjustments, and the interactions between that and tax/billing periods/etc makes the entire thing a headache.
And then you often need to add some way for the sales team to change things on an ad-hoc basis for sales/special negotiations/etc because of course big customers are going to negotiate something special, and that's all custom logic. After that, now the marketing wants referral credits, and then they want promotional periods that interact with customer loyalty, and on and on it goes.
Every time I didn’t, I’ve wished I had.
Simple things like your user just did X, in this country and at this time. This has a tax implication
We got in Call Data Records (CDRs) every day from BT, which we then had to calculate billing for based on our business rules, which had to be somewhat aligned with BT's pricing so we'd actually not lose our shirts.
It was pretty straightforward with one price between 8am and 1pm, then a cheaper one between 1pm and 6pm, then calls were free from 6pm until 10pm as long as they were less than one hour then billed at an even lower rate, with an hour of free any time minutes that reset at midnight. So, if you never made more than one hour total of calls between 8am and 6pm, and kept your calls to less than one hour the rest of the time, the actual call were free, right?
You would genuinely be surprised how many times the billing system they were using would toss its cookies because someone made a phone call at 1740 (billable), continued right through 1800 (not billable) to 1900 (billable after 1 hour but there's still 40 minutes on "anytime" left so logged but free), right past 1940 (billable again) and on past midnight (free for another hour until anytime runs out).
No, you can't just kick someone's call off after an hour.
> No, you can't just kick someone's call off after an hour.
Agreed... but I worked with a telco that did exactly this!
They are indeed no fun. And payment APIs make me want to shoot myself. Whoever writes them seems to enjoy making end developers bleed from the eyeballs. But I've written my own in-house billing software three times over since 2002 when I had to write my own front-end API to a raw CC gateway and guess what the outputs meant ... it's much simpler now to deal with asynchronous payments and it's not the end of the world to manage them.
The much, much bigger problem is getting your clients to pay on time. Knowing who usually pays and who's okay being late (versus who is habitually delinquent) and keeping tabs on that before you bill them again so you don't have to suddenly confront them about canceling their services. The smaller the business, the more significant a portion of time this can take up.
When I started working for an ad agency in 1995, the owner had me stand outside a guy's factory next to her with a picket signs because he owed her $5000 for an ad that ran but was a month overdue. I have written backdoors into most large pieces of software I've built, in case of nonpayment; but never had to use them. I've pulled the rug out from hosting and left people's websites saying "Joe X doesn't pay his bills" after being treated badly by clients who were 6 months late paying. These things all give me a certain sense of glee but never get me paid.
The best strategy I've found is calling your clients on a rotating basis and checking in to see what they need next and gently reminding them that their bill is due. I try to make this round about once a month.
> To solve this problem at scale, we've adopted a radical approach: we're building an open-source billing API for product-led SaaS. Our API and architecture are open, so that you can embed, fork and customize them as much as your pricing and internal processes require.
One key thing to remember is that if you want your billing to make sense to your customer you can't get too clever about it. A few rules of thumb: when you have to round anything, round in favor of the customer. Split the logic that calculates what should happen and the billing part of the billing system. For example, if you store a next_billing_date for each customer you can trivially adjust the date as needed, but if you have a crobjob that calculates which customers should get billed and then immediately bills them manual overrides are harder.
Giving a user what amounts to a few free days of service when they upgrade/downgrade is no big deal. Especially because you should see more upgrades than downgrades in any given month. In some cases the easiest way to credit people is to move their next_billing_date forward. For usage based pricing (e.g. number of active users in a month) do the easy and fair thing where you take the minimum of active users during the month (store num_active_users each night for each customer, and use that for billing and for user facing billing dashboard). Otherwise you get edge cases where you overcharge customers when they add and remove many users because then the max of unique user ids is >> the number of active users at any given time. The example query in the blogpost gets this wrong.
The SQL query that aggregates the number of API calls for a user is wrong for the same reason because it confuses API calls with billable API calls. You should have a cron that counts the API calls for each user and stores them separately in a billing table. Then when a good customer has a junior programmer make a ton of API calls by mistake you can easily go into the billing table for that customer and manually adjust the usage, with total confidence that future invoices will be correct. If you compute the usage at the time of billing then everything gets needlessly complicated.
Side note: Is "I don't want to write/use a billing system" a compelling reason to release software for free?
I think indeed some users will use us for free and never pay, whereas larger companies actually want to pay to get the hosted product, custom features, SLA, etc.
We are still in the early days, but here is how we think about our own pricing : https://www.getlago.com/blog/how-we-think-about-our-own-pric...
The first paragraph ends with „your tax decision tree should look like this:“ and the second paragraph starts with „Now, imagine that you sell different types of goods/services to.“
"chargeable items" are things we want to charge for. In my experience there are exactly three kinds of chargeable item, which operate largely in their own domain:
- "usage charges" are charges based on usage of some resource (eg, bytes, time or phone calls).
- "recurring charges" are charges that are billed on a regular basis, e.g. a monthly fee. Recurring charges get complex due to the need for pro-rata, e.g. if your customer is on a monthly plan but signs up in the middle of the month, or changes plan in the middle of a month. (you can get away without this, but that just moves the complexity somewhere else - and computing refunds can become a huge pain at scale, especially if you want to build a system that is 100% self-service).
- "one-time charges" are charges that are raised due to actions taken during the billing cycle; you might also call these "event charges". Things like connection/disconnection fees etc that are generated by a process or external system. These are the simplest, just an SKU and a value, maybe a qty and rate if you need it.
Arguably, the most challenging domain for billing in telco circles is "rating", which is the process of computing the usage charges for individual charge detail records (CDRs) and aggregating them so that your database doesn't get overwhelmed. You might get a billion CDRs in a month. Although not all billing applications require rating, anything that does usage billing is doing some kind of rating. In Telco we also talk about "mediation" and "guidance" but that's another story...
What's important about the above is the knowledge that there are different charging domains which need to be handled differently. There's a huge world of difference between a recurring charge and a usage charge.
"billing" is the process of collecting these chargeable items together, to prepare them for sending to a customer. The resulting database rows might include the raw pricing along with tax, discounts etc. The important thing here is that we have a means of separating what we want to bill, from what we actually send to the customer, because the timing of an invoice is different from the timing of the charges.
"invoicing" is the collection of the outstanding chargeable items into a final invoice, which can actually be sent to the customer. Interestingly, once you've issued an invoice to a customer, modifying the amount owed is probably illegal in most countries, and most certainly a bad idea.
"payments" is the process of receiving money from credit cards.
"collections" is the process of asking people to pay (It's also called "dunning", after the famous debt collectors, Dunn and Bradstreet). It arguably also includes policy actions like restricting access for non-payment, raising late fees, etc. Another can of worms...
Anyway ... you can see a kind of billing pipeline here, where you have an ingress of chargeable events, which get rated, aggregated, held for invoicing, and then an invoice is raised, payment is requested, and non-payment is processed.
This is just based on my experience, the industry seems to vary somewhat on the specifics. And ... it seems easy enough when I write it like this, but there are many devils in the details. Nevertheless, I hope this helps someone.
You mention usage and one time charges as being distinct. Ive generally seen them treated as the same thing; “instant”, “action”, or “usage.” Effectively atomic units of metered usage that are aggregated later, and the corollary to recurring usage. Is there some reason to not model “one time charges” as just a another (infrequent) variant of usage. Eg a “service connection” usage with value = 1?
I didnt see you discuss pricing in your list. Do you normally treat pricing as part of usage? Ive always see it as a separate value thats joined with usage per customer-billing period.
For usage, as you say, there is a complex process that’s needed to compute the final value for an invoice - in telco the formal names are mediation, guidance and rating - which isn’t needed for one time charges. A usage charge may be the aggregate of thousands of usage samples (CDRs, charge detail records), processed through a bunch of rules. The CDRs can come from many sources and be formatted differently. Importantly, you don’t want to treat these CDRs as full charges that appear on an invoice, because doing so will overwhelm your billing database for no benefit. Similarly, while you could in theory send a one time charge through the rating process, doing so is probably wasteful, as the processing functionality doesn’t significantly overlap between usage and one time charges.
Pricing is very complex! You are absolutely correct, it should be separate, and it should apply to all three types of charge (consider that the actual decline fee charged to an account might be different depending on what plan they are on, or some other factors). So yes, pricing ideally needs to be separate, or at least decoupled from the billing process. That adds another layer of complexity for sure.
I hope this makes sense, It’s getting late where I live :)
Edit: belated additions to your great list of “basics most people miss some of”: pro rating & amendments, especially retrospective prorata! And the classic retrospective amendments after you’ve crossed a billing period. Aka “we’re sorry and here are some fungible credits!”
Pro-rating is a real spanner in the works! If you’re half way through a month and you’ve already charged $100 for the “silver” plan, and the customer wants to upgrade to “gold”, you need to refund the unused half of the month on “silver”, or somehow take that already invoiced amount into account during the upgrade process. The only alternative is to give “gold” away for half a month (which opens a path for gaming) or to wait until the end of the month to change the plan (pisses the customer off).
Calculating all this accurately is possible, but very hairy, and the solutions we came up with after many years in the space were non obvious. It’s far from rocket science, of course, but it’s far from straight forward.
Now comes the unexpected part. People kept asking us why we have only a single plan. Maybe all of us are so used to multiple pricing plans on saas products. So we had to explain ourselves in the pricing FAQ’s.
Are you Indian, perchance? How you write prices in India doesn't work in other places; this looks strange to someone who works in nearly any other currency.
Any of the following would look a lot less out of place:
$50 50 USD $50 USD 50.00 USD $50.00
Of these, I recommend the first one; dollars default to USD worldwide unless specified otherwise.
If there are invoices involved then it is going to be fun :)
I think that this article is good in that it provides a significant amount of detail about many of the complex problems of billing. I don't agree that everything in the article is a big deal, but it certainly calls out a bunch of the things you have to think about, and it's a long, long list. And it's also just scratching the surface - some of the questions lead to many more questions, and significant complexity.
So, if you don't architect your billing platform in a way that properly separates the high level moving parts, you'll eventually end up with a big pile of spaghetti. Which is bad form for something that's so critically important.
That's not to say that you have to implement every single possible feature, or even that separating things is particularly hard. It's possible to build a simple billing system with just a few SQL tables - but they are probably not the exact tables that you are thinking of. But it does means that your billing platform needs to make as few assumptions as possible, and be structured in a way that leaves room for change in the future.
That's because billing is critical to a business' success, and it therefore requires lots of flexibility. One way to find product/market fit is to do your billing in a way that better resonates with your customers. I mean ultimately that's what I think AWS did to colocation. So if you can't keep up with the demands of your sales team or your customers, you're going to be in a bit of trouble.
As engineers, we don't get to set the rules for billing, because that's not how customers work. The billing system needs to support the requirements of the business; not dictate to them how they need to do things.
Anyway I wanted to add a couple more things to TFA, just to illustrate that there is even more to it than they've listed.
Policy management - what do you do when a user spends too much before the invoice is raised? Do you send an email / sms / restrict the service? how do you avoid doing this multiple times in a month? how do you integrate the billing policies with other policy systems such as usage policies? This can get really deep...
Invoice rendering - very few people seem to understand that invoicing is the one time every month that you have the legal right to send unsolicited emails to customers. But in my experience, invoices almost always look like crap. What a waste!
There is heaps more to this: how do you quickly fix billing problems, such as an incorrect plan? how do you backdate changes? what's your pro-rata policy on fixed charges? what do you need to integrate with the finance platform? ... the list goes on.
So please forgive my shameless plug - I'm available for consulting on billing systems any time. I really know what I'm doing. See my profile for contact details.