Show HN: An API to store balances and transactions | Hacker News Reader

Show HN: An API to store balances and transactions

151 pointsandriosr102 comments

Hey everyone!

I'm Rob and I want to share what I've built - a solid ledger behind simple API: https://decimals.app

The ledger is a key piece of any financial system and is easy to get the design wrong, resulting in troubles for making sure money movements are adding up.

I took a lot of inspiration from good designs from Square, Uber, and others for the scalability and security aspects. And on the usability side, I applied many concepts from the Ledger CLI, making it very easy to use.

I'm really happy I get to show this to you all, thank you for reading about it! Please let me know your thoughts and questions in the comments (The sample code on the landing page is not working, I'm fixing it).

[1] https://www.ledger-cli.org/

[2] https://www.youtube.com/watch?v=iN6mhI5hFt4

[3] https://developer.squareup.com/blog/books-an-immutable-doubl...

[5] https://docs.decimals.app

View on HN

I actually pitched this exact idea to YC and was part of W20 - but I very quickly pivoted away from it.

If anyone is wondering about this, the basic business challenge you have to solve is being in the right place at the right time. This kind of thing is very sticky once its in place, and most startups don't realize the complexity in the domain early enough for a stand-alone service to make sense to them.

I think as far as startups in this space go, the best path forward would be to nurture an open source project in the long term.

I ended up open sourcing my work after the pivot here: https://github.com/affiga/scaledger

Interesting! I would love to talk to you about the experience, can we have have a quick chat?

Yep, email in profile

And it looks like a similar effort that was abandoned at subledger.com. It think there can be a market for something like this and hopefully this one works out.

Thanks so much! Didn't know them, I will reach out. Do you know if they completely shut it down?

We pretty much use a private blockchain for the very same purpose.

Its internal, only our systems have access, each user/customer is auto created an account on signup, we hold the private keys encrypted with the user's password so we can't access the key itself, keeps track of everything with 100% consistency.

1s block mining and raft consensus settings so pretty much instant and low resource utilization...and massively scalable.

Sorry to say but if I am in any type of fintech, I would never ever outsource keeping track of my customer's account balances and source of truth.

Completely understand. This is a normal concern for new technology, not every company is an early adopter.

The same way Lyft would not even consider using Stripe in its first two years, considering how critical credit card data is for them.

I really like the design you have internally! I would love to hear more about it, can we talk?

Curious, why is card data important to Lyft? How does card data benefit a transportation business in any way beyond just taking money from the cards?

If they sent the card data from their customers to a 3 person startup it could be stolen and make their customers unhappy.

This was likely just as much about credit card fees as it is about data.

Stripe's amazing service also comes at a substantial premium. For a high-volume thin-margin company like Lyft where every penny counts, per transaction fees at 2.9%+ are killers. Furthermore, large companies prefer to own the raw card data themselves as the interchange rates for card not present debit cards is different than card not present credit cards. Once you hit a certain amount of scale this .15% difference is substantial.

(edit: I missed that parent comment was about trust in a 3-person startup, but point is still somewhat valid)

Companies at even a hundredth of Lyft's scale aren't paying 2.9%+ to Stripe, they're on cost-plus pricing based on interchange fees.

If you're using a private blockchain where only your systems have access, why not use postgres with restricted user permissions?

I can't answer for OP as I don't know his/her situation, but every time I have asked this question to other companies, the answer boils down to

"We really wanted to use a blockchain and it is the main reason why we build the company in the first place so we never questioned that choice".

Am not the parent but, for example, the Bitcoin blockchain is essentially a double-entry transaction ledger. Every transaction has to balance inputs and outputs.

If the product poster is using has that property, then perhaps is a reasonable choice.

Postgres does not have any ledger balancing primitives.

For Decimals I’m using a hash-chain. So, entries are always balanced, atomic and immutable. Not a blockchain, as there is no need for consensus, but the properties for checking integrity are all in there.

Bitcoin-like blockchains are by default triple-entry accounting: https://onlinelibrary.wiley.com/doi/abs/10.1111/acfi.12556

I may be misunderstanding but what happens when your customer changes their password?

Ideally the login password hash and the account for record keeping's password would be decoupled

Hey Canterbury, very intrigues with the idea of using a private blockchain for this purpose. I building exactly that as a service. Could we have a quick chat about what issues you had and the process?

This is pretty cool, I'd wonder how it compares to an open source accounting library like double_entry. I guess I'd be hesitant to integrate with an external service if I could accomplish the same thing with a library.

Have no idea if there's a market for this (I'm certainly not a customer), but something none of these libraries are set up to do is convert these balances to erc-20 tokens. The combo of a SaaS with a simple API like this and a trusted mint to sync balances to erc-20 tokens could be interesting.

[0]: https://github.com/envato/double_entry

I would check out https://frappebooks.com/. They are building an open-source accounting system like Freshbooks. It can also be used for personal accounting like Mint.

Interesting, thanks!! I know Frappe, it's from the creators of ERPNext. They are more like a full-suite accounting tool, like Mint, as you mentioned. Decimals.app is a general-purpose API, it can be used for accounting and other use cases if you need to build them and can't use Mint and others.

Very interesting, thanks for the feedback! I found a couple of similar projects, but not this one, I will definitely check them out. Thanks!

I'm also interested in these systems as opposed to implementing accounting systems internally myself. How do all of these compare to https://hledger.org/1.2/hledger-api.html?

I can’t count how many times I’ve individually solved this problem for various companies, and fixed broken ledger systems. I’m currently dealing with the terrible legacy decisions of single-precision float database columns leading to floating-point precision loss on a multi-million $$ eCommerce platform.

What a schlep.

I’m curious, do you store currency values as decimal dollars or integral cents?

The code sample on the website indicates integral cents:

    // Transfer $10 from Alice to Bob
    decimals.transactions.create({
      from_account: 'Alice',
      to_account: 'Bob',
      currency: 'usd',
      amount: 1000
    })

Yep, as @zomglings and @vangale noted, always an integer. Thanks, guys.

They can represent anything you want, cents, mils, and so on.

Ironic that the app name is "Decimals".

Personally, never liked the apps that used integers for what in the real/human world are decimals. I understand (sort of) why they are used from a tech perspective. But ultimately, tech is used by humans.

It's an issue with floating point accuracy usually...

2 point decimals can usually be accurate but given enough operations, you can sometimes get a weird floating error and rounding / clamping is discouraged in banking.

He's talking about fixed point, not floating point. The decimal type in SQL is an exact, fixed point numeric type. It can avoid loss of precision, like integer, but without the scale conversion step.

That's what we call "an abstraction" and that's what they were invented for.

At some point, in tech you are going to have to create an abstraction at some layer. The upper the layer, the more the abstraction should resemble what humans see as "normal".

Did you know all numbers are stored in "registries" in the computer memory. Is a registry a number? NO! It's just bits.

At what point can you actually treat a "variable" as an actual "decimal"?

How does a storage system actually save a decimal number? Not sure, and up to some point I don't care, as long as I know the accuracy and trade-offs I'm getting.

I guess my point is, using integers sets to represent a decimal number is okay if it is coping with the lower layers' limits. If a lower layer already provides a proper solution, then there is no need to provide the same.

But ultimately, abstractions are what matter and what has allowed us to progress. All you need to know about an abstraction is its limits and compromises.

When I run the example code I get "Error: Cannot find module 'decimals'". I'm on latest Chrome macOS

Noted, I'm working on it. Thanks!

Not to be "that guy", but if your pitch is that you can keep money transactions consistent, also make the use of semicolons on the landing page example consistent. ;) Apart from that it looks really nice, but as a naive person it would be really cool to give examples of issues i would run into, if i tried building something like this myself...

No worries at all! I'm here to get feedback and thoughts. Thanks so much for noticing and letting me know, it's now consistent! It makes a ton of sense to add the common issues section, it should speak to people having the problem as soon as they see it. I will work on that.

I would definitely second that. I don’t have a great grasp on why I shouldn’t do this myself. Somewhere (it doesn’t have to be on the landing page), I’d like to hear about why I shouldn’t do this myself.

Got it, I will improve on that. The point is: besides being a consistent store, Decimals connects to multiple sources of data and make it available in your ledger automatically. Say you have transactions in Stripe, besides building the system to store the transactions in-house if you need, you also have to connect to their webhook to pull the data. Then you need to compute the usage of your saas, you can also do that on Decimals, just use the currency of your app (MB, GB, events, etc), now you have payments and usage in a single consistent store, and so on.

Having gone through the pain of writing code like this in the past, this looks amazing. How are you planning to charge for decimals?

In your code sample, you use from: 'Alice' but list transactions for some kind of complicated balance ID. Where did that ID come from?

Also, your RunKit sample is not working since they don't have the decimals package available (assuming it's published on npm already).

I'll add a note to the post, thanks! Working on that now.

Great question, each Account holds a list of balances, one for each currency.

When you first use a given currency on an Account, a Balance is created along with its id.

The ID comes from the list of balances of the Alice Account.

Is there a way to access that balance ID programatically? Right now, it is a magic string and it isn't clear to me how I would get access to it in my program. Perhaps from a return value on decimals.transactions.create?

Also, shouldn't decimals.transactions.create be asynchronous?

Pricing is similar to the Segment.io model, you pay for the number of accounts, in chunks of 10k.

You get the most value when you do more transactions, so that's why we don't charge for that, we don't want companies limiting what they to with Decimals to save money.

Also important to note is that this is not one more layer in the payment cake taking a percentage of your revenue, the pricing is based on API usage, with no relation to your revenue.

Great, how long before I can try it out?

You can try it today! Where can I reach you?

Email in profile.

Not showing up here :/

Sorry, didn't realize email field was not visible. Updated profile.

Looks pretty cool and useful.

Only note is that the runkit environment doesn't seem to be working. "Error: Cannot find module 'decimals'"

Thanks! noted, I'm working on it.

Why is something that should be a library over an ORM or other data store now a service product?

Thanks for the question! My view is that when you are creating a product and there is something that can get you faster at delivering your core value to your customers, it makes sense to leverage it.

From past experiences, on a reasonably big company (>100 engineers) that needs to do these money movements checks, it would save at least 3 engineers from support/evolution and not counting the development time for a good ledger.

From my experience as running my own SaaSes, I'd rather have a library over an ORM, or a self-hosted service. Having yet another cost to manage, another API to monitor, and an unreliable interface (HTTP over internet), just isn't worth the cost.

That is very interesting to hear. I would love to better understand the kinds of problems you may have faced with these systems.

On the cost side, the goal is to charge on average 1/10 of what you currently spend, so cost is actually a feature. I'm working on a self-hosted version for some customers that are asking for it, but not due to reliability of HTTP over the internet, I would love to hear more about this. Can we chat?

Cool. Couple questions:

1. How do you handle / plan to handle multi-currency accounts? (normalize to single currency vs. Selinger's approach https://www.mscs.dal.ca/~selinger/accounting/tutorial.html) 2. What tech did you use to build all this? db/languages etc.

Great questions, thanks!

The API is currency agnostic, so one account can have many balances from multiple currencies attached to it. However, you can't do transactions cross currencies at the moment. You can do it on your side, and use the API to store the result, both normalized and Selinger are doable. I definitely plan to add this at some point, thanks for the reference, it's great!

On the tech side, the service is written in Go, and the database is DynamoDB, everything runs on AWS right now. I'm planning to add support for Cassandra and MongoDB, as some enterprise customers want to run it inside their datacenters.

Money and keeping track of it is a big problem. Depending on your problem though, its not fully always in your control. For example, we integrate with a discount calculation service that only supports values up to the 100ths. This means that we have to introduce rounding as an intermediary and then solve the "off by one cent" problem, after the fact.

In addition to the problem @joshuakelly mentioned, I think one of the challenges here will be to gain trust. Asking someone else to keep track of your books is a pretty tall ask. How does it affect a given company's auditing? What about SOC Compliance etc? I would suggest you try to address some of these things.

Best!

That's great feedback, thanks! I agree with both points, handling different precision strategies from multiple parties is a challenge on itself, we don't solve it yet, but is definitely on the roadmap. On the trust side, it's a question of time, as the curve of adoption for any technology, we have been working with the early adopters now, and providing them the competitive advantage they deserve for being innovators. Thanks for the comment!

SubLedger was another startup in this space a few years ago, which I thought had the right idea. But they didnt really follow through. They seemed to focus on uber and their ilk as customers, where as I felt they would have been a killer if they focused on being the accounting backbone between business apps. Like Xero is, but api first / only.

https://www.crunchbase.com/organization/subledger

Interesting! I’m going to try to learn and avoid making the same mistakes. The goal is definitely to be the backbone of any money movement record. The goal is to do what Segment does for analytics data, or Auth0 does for identity, but for payments data.

Yes, in the end there are only lists of transactions and so great potential for interacting with them from a central, simple API. It may or may not be worth having a look at the open banking APIs that are in use here in the UK.

Thanks! I read a couple of specs, didn’t like the design, I think it’s too complex. But it may be due to the complexities of the UK banking system. From an integration point of view, it’s definitely something I’m looking at.

I thought this sounded familiar. I got an email out of the blue 10 days ago about this app from the creator:

> Hi!

I run a startup trying to make it easier to create systems that store transactions and balances.

I found your contact searching the stargazers of the php Money lib: https://github.com/moneyphp/money

Are you using it?

The project is in the early stages, I have nothing to sell. I need help to make sure I will build something useful.

-- Rob.

Hello Ian, yes! That was me. I reached out to a couple people using money libraries to better understand how are people dealing with the problem right now. The goal was to help in the design of the api for client libraries and get a better sense of how the product fits into different orgs. Thanks for commenting!

How would this be better than designing a simple SQL schema, and then generating the ledger as a report?

Good question, I think it is the same asked here: https://news.ycombinator.com/item?id=23355547

Let me know what you think. Thanks!

I think that the people you are targeting (developers) do not care or know about the issues you are solving, and they also have the skill to make their own thing - its too close to "persist this list of numbers, sum them". Using decimals, how would users generate reports?

I think maybe you should do a detailed blog post comparing the difference with the default course of action developers take when they need to record transactions with the benefits decimals provide.

At a loans company I worked at before, we just used a transactions table and generated accounting reports using SQL.

That's great advice, thanks! What sources of data were you pulling into your transactions table at that company? For instance: payment data from Stripe?

Well we weren’t really pulling data as our database was the “source of truth”. We had a schedule to charge cards, so the state machine was defined completely in our code - unlike Stripe-like companies that control the state machine and send you events to tell you what is happening.

The real answer is that double entry is a different problem, and SQL/relational databases are not ergonomic for solving it.

Most SQL ledgers are just a table of transactions. What happens when you have an error/typo in a transaction? SQL databases will ensure that your error gets safely committed to disk, with zero built in machinery to detect what double entry knows to be an error in the data.

(I don't know if OP actually solves that problem in their service, but this is the key idea in double entry.)

Cheers.

A string field typo shouldn't be causing your production system's ledger to become misaligned or get into an inconsistent state. To prevent that, you use some form of unique IDs and have constraints on your data model. Double-entry transactions should be done on top of a database transaction so that they're atomic.

The semantics required for double entry are essentially that multiple records are being added to different tables, and there are math rules that have to get applied to the values in specific fields across those tables (e.g. to ensure that the individual account transactions balance). It's not just an atomicity problem, there is a correctness issue specifically around the data.

These rules are not expressible in what SQL databases consider to be field-level constraints. They can be expressed in triggers, but doing that is the path of pain.

Those are all application concerns which you should get right and no database out there can do that for you short of exotic ones that are purpose-built for this domain. You specifically said that "[SQL has] zero built in machinery to detect what double entry knows to be an error in the data" And I mentioned that's not the case because double-entry is solved by mostly two things: Zero or two transactions happen on separate accounts, not 1. And that the separate accounts happen on uniquely identified accounts (and not by string-identifiers that can have typos), which is solved by unique & foreign key constraints. The other side to that is that the sides of the double-entry have to be identical in amount. There are no "math rules" beyond that, but again this is an application concern.

Relational databases are perfect for solving a ledger problem, and as long as you build a solid application layer above it to solve the domain problem, you won't have any issues. I understand there is a point that this thread is making that "block chain" and "NoSQL" DBs are somehow better for modelling a ledger, but that's inherently false and the same criticisms you apply to relational databases for this, you can just as easily (if not more) apply them on exotic database types.

With double entry encoded in software, wouldn't the typo just get inserted into two places instead of just one?

The rules around how a single high level transaction gets divvied up into what ultimately need to be balanced accounts can be complex, with derived amounts being calculated, opportunities for rounding errors, etc.

I've never done any double-entry accounting work before, but wouldn't it be sufficient to wrap the credit and debit operations in a single database transaction to ensure that either both are committed, or neither? maybe I'm misunderstanding what the problem is. Genuinely curious! I've always wondered how these kinds of things are implemented at scale.

Well, no- not only do the credit and debit (logical) financial transactions have to be committed in the same (physical) database transaction, but more importantly, the amounts have to balance. A SQL database doesn't have any opinion on those kinds of semantics.

You can of course- and people do all the time- encode those kinds of "business rules" into application code, or into database code like triggers, but:

1) the abstraction level you are operating at in the double entry domain is around balanced accounts. Having a "ledger database" that provides that kind of abstraction to your application so that you are not solving for balancing transactions all over the place in application code is very important. Important for the same reason that using a SQL database saves you from having to worry about two phase commit and transaction logs and so forth in your application code when saving sensitive data. The guarantees a SQL database offers are lower level than what one needs in a ledger database.

2) to your point of "at scale"- at scale a double entry system often accepts some kinds of unbalanced transactions, and then has workflows to support various kinds of reconciliation. A lot of this reconciliation is time based- every bank to this day at least does reconciliation at the end of each business day with "close of business" batch processing jobs. Some can also do it intraday. At scale means a lot of additional actual "business rule" complexity.

Hope that helps?

Yes that helps thank you!

Completely agree! I've seen SQL implementations of ledgers fail quite a few times.

I'm using DynamoDB, but any no-SQL database will do the work (currently working on Cassandra and MongoDB connectors to enable self-hosting for enterprise customers)

On the implementation side, it's a mix of both: I'm using the conditional insert/optimistic locking of no-SQL and modeled the ledger to be a hash-chain at the application layer. The design is actually simple, and it scales indefinitely.

Besides that, I'm using DynamoDB streams to build roll-ups of the transactions and make sure that all transactions are always balancing to zero in real-time.

Cool, got it.

Are you actually seeing demand for on-prem hosting? Serious demand such that you would lose the sale from a customer you cannot live without? Completely different support model and neither Cassandra and especially Mongo are anywhere near Dynamo.

I also have seen plenty of cases where balance to zero only at end of day is feasible, but real time is a good way of picking the right customers. My completely unsolicited advice- do the same with the DB, stay away from customers who say they need prem. But I'm just a schmo on the internet. Very best of luck to you!

A couple of companies asked for it, but I couldn't agree more with you.

We are in the design phase of these implementations, as there is a lot more than code to consider for a self-hosted version.

Thanks for the advice, it's great to get your view on it.

DynamoDB is the wrong technology for this. You can't prevent phantom reads. Transactional databases (precisely invented for this!) are the way to go. To quote DDIA:

'A service that allows users to spend money or points needs to check that a user doesn’t spend more than they have. You might implement this by inserting a ten‐ tative spending item into a user’s account, listing all the items in the account, and checking that the sum is positive [44]. With write skew, it could happen that two spending items are inserted concurrently that together cause the balance to go negative, but that neither transaction notices the other.'

Thanks for the comment! It’s a common misunderstanding that no-sql dbs can’t be used for this, but any would work. Dynamo dB have consistent reads and conditional inserts, which is sufficient to build an optimistic locking system that ensures no double spent money.

No problem. You should look into this more. Conditional expressions in DDB do not provide aggregate expressions and can only determinate on single attributes belonging to the same table.

Consistent reads aren’t enough.

You would need to prevent phantom reads. DDB doesn’t support range locking since it’s a LSMT. You can contact the AWS solutions engineers or perform a quick online search: ‘However, neither DynamoDB nor the library support locking for range queries, thus phantom reads can potentially result in phantom reads.’

https://dbdb.io/db/dynamodb

I'm actually building a crypto trading system from scratch (like gekko but more sophisticated). For now not as a SaaS, but even with a simple system, all the transactional problems occur quite fast. This is really interesting !

That's great! One of the companies using Decimals is building a crypto exchange, it's been great to see how the API can be extended. They are creating a multi-signature wallet on top of Decimals, adding a signature to each transaction. Let's talk, I think it can really accelerate what you are building.

I mean, I'm on the client side, so I don't need a ledger to manage wallets, for now. But as the system evolves, I'll definitely need to 'allocate' budgets in the form of ledgers, where an algorithm decides to move funds from one account to another.

Hi,

I've signed up for early access and tried replying to your emails but I am unable to reply to your decimals app email address, I get mail failut. Please reach out with another email, could really use this.

collin at waystocap

We are planning on offering a competitive service to this later on this year. The major difference for us is that we’re an actual bank, so we need to run this infra whether you pay for it or not.

Interesting! That's great to hear, I think there is a big market for this. A lot of companies are struggling to build this in-house, without the required finance/accounting domain knowledge. I would love to chat, can I reach you on keybase?

Why would you not use an accounting package like Xero to manage this?

Xero is an accounting app, and it has a ton of features. This is a simple API to validate money movements, it can be used as one of the pieces to build a dedicated accounting, billing, and other systems if existing solutions like Xero doesn't work for you. It's general-purpose, where Xero is more a dedicated app for accounting.

Thanks for the insight. Appreciate it.

Xero user here. Would leave in a second if I could. Why:

- Next to nothing in terms of expense graphs.

- Terrible API for polling for alerts based on reports.

- Saving custom reports not really a thing.

- Mobile app is hot garbage.

- Bank feeds drop a lot.

- My accounting firm not a fan.

- One light blue skin forever, like being relegated to a blue hell.

- Integrates with things like Zapier & Integromat, but not deeply.

- Parsing JSON reporting via Integromat is pure pain.

This is interesting! I heard a lot of complaints about the Xero API, it has low rate-limits and you need some workarounds to push data in at a fast enough pace. Definitely not the system to be used at scale.

One thing that is in the roadmap is to bring data from Xero so that you can interact with it in Decimals. It should give you a much better API, and we make sure it's going to get synced in the background. Let me know if this is something you might be interested in getting early access.

This is a great idea, and I think you should pursue it.

Immutability of transactions is obviously a fundamental necessity and coupling this with GraphQL seems like a great fit.

Thanks! Yes, it is, I will make it more clear on the copy.

How did you build the landing page? (any tool?) I like the sleek style, it reminds me Stripe pages.

Hey! I used one of the https://cruip.com/ templates, and the sample code component is from https://runkit.com/home

Thanks a lot! Good resources!

didnt ledger cli try this with dot ledger that was used by hackclub?

Nice question! Dotledger is a full suit accounting tool To replace Xero personal finance. Decimals is a primitive API that can be used to build that or other kinds of applications, but it would be only one of the pieces of the overall system at the moment.

Reply on news.ycombinator.com