Beancount: Double-entry accounting from text files
github.com
github.com
The notion behind "accountants don't user erasers" is that like a blockchain, you're creating a trail of evidence. If all of the data is mutable all of the time, its evidentiary value is low (for example, the IRS gives more weight to "contemporary" evidence than to records constructed or altered after-the-fact).
Another reason to freeze, snapshot, or close your books is to not lose the support and detail behind any financial statements you've produced. Imagine getting a business loan based on your Balance Sheet and Income Statement but later being unable to reconstruct how those were produced.
Generally, accounting systems use a system of cumulative, immutable journal entries. If a mistake is made, then a reversing or adjusting entry is made.¹
There is merit to text based bookkeeping, but don't create a world that is so free form and mutable that it loses it meaning and connection with the real world.
Basically the idea is, as a software engineer, I believe accounting book can be automated like how we build software. There are many interesting workflow we can bring into the system, such as
- Pull request review - Commit sign off (pgp signature) - CI for generating reports
and many other interesting aspects. Also just like software, I believe many of the orgs can benefit from open sourcing their financial data, like no-profits or government. So I am thinking making open source accounting book free repository for the product just like GitHub did previously. Anyway, I just found this thread happen to appear on hacker news so just want to share the stuff I am building. Feedbacks are welcome.
The first paragraph of https://www.cfo.com/accounting-tax/2007/10/finance-in-histor... explains:
“Bean counting” has long been an insulting term for what finance professionals and accountants do. Often it’s been used to tar CFOs as transaction processors — a role largely relegated to the back office. What’s more, people like to use the phrase to ratchet up the pedestrian aspects of finance by tagging practitioners as “mere” bean counters or “little more than” bean counters or “simply” bean counters. Such clichés get tossed around a lot.
Per https://www.phrases.org.uk/meanings/bean-counter.html , "A disparaging term for an accountant, or anyone excessively concerned with statistical records or accounts."
It's VERY hard to accidentally miss capital gains, or get cost basis wrong on something, or have something not balance. Much harder than even ledger-cli, let alone the other alternatives. It feels at times like working with a static type system versus a dynamic type system.
And the burden associated with this is not anything more than other tools. Importers have been great. I have 68 asset subaccounts between my spouse and I (if you do double-entry accounting you'll know this isn't actually that many), and I spend about 1-2 hours a month balancing the books, doing reporting, etc.
Finally, it's quite easy to get your transactions into a pandas datatable or similar, allowing you to utilize your programming/datascience skills to do things that just aren't available in other tools. Mint offers nothing here, spreadsheets can do some of it (but the downsides of accuracy), and gnucash requires learning some niche scheme stuff; it feels very "tacked on".
There is also piecash, which gives a nice Pythonic interface to the SQL files generated by GnuCash: https://pypi.org/project/piecash/
> Note this feature is considered experimental. It works for most of the common use cases but some corner cases have been reported to result in data loss. [1]
What importers do you mean? Are these importers you've written yourself, or existing importers you were able to reuse? What format are these importers pulling from (PDF statements, CSV, OFX, etc)?
https://github.com/jbms/beancount-import
https://beancount.github.io/docs/importing_external_data.htm...
Support for OFX is generally great, CSV is also great. There are some proof of concept things floating around too for PDF.
I'm asking because I've generally come to the conclusion that PDF is the only universal data format that every institution will offer.
Of course I've submitted requests pointing it out/requesting it, but to be honest I can only assume it's deliberate.
CSV feeds are often surprisingly useless for programmatic / feed use, they're often littered with stupid textual headers and footers and lack important columns - they're designed to be opened in excel and looked at by a human, not parsed and ingested.
- debit card: merchant name + date, card number, blank
- alternative card-based payment system: constant (account number?), merchant, constant
- alternative app-based payment system: constant, merchant, user custom reference
I've never used any import tools though, I just enter the transactions manually maybe once or twice a week. It's been fine - especially using beancount-mode for emacs, although admittedly it's just my accounts, it might not scale so well for a family.
I really prefer having complete control of my financial data this way, and reports can be as custom as you'd like. Better than something like Mint in my opinion.
https://plaintextaccounting.org/
Previous discussions here: https://news.ycombinator.com/item?id=28420797
I do see the demo has an editor!
Me too, one of main reasons for choosing it.
> Is there a tool to convert brokerage data into Beancount data from scratch?
Problem is they are all different. But other people share what they have come up with. Maybe check the mailing list, also search for Red's Importers. If those do not work for you 'out of the box', you might need to hack something together. But you can get help on the mailing list.
* Turns out they address the comparison: https://beancount.github.io/docs/a_comparison_of_beancount_a...
What I’ve been doing lately is just generating ledger-formatted ledger on the fly for this that I used ledger before, while still benefiting from the strict beancount format, balance assertions and such that ledger doesn’t have
It appears ledger does have balance assertions.
It seems though that the assertions are attached to transactions. The way I use balance assertions on beancount is that, say, payday after balancing everything I'd enter
> 2022-01-30 balance Assets:Bank 500.00 USD
and that will ensure that whatever happens, even if I need to go back in time to clear transactions, the amount needs to be that one, and that balance is usually straight from my current bank balance.
So, it seems slightly different use case that ledger's balance assertion
Ledger's simple model makes a lot more sense to me (ie. everything is just a commodity), but it sounds like beancount is set up to deal with a lot of the extra bullshit that exists in the legal system.
https://github.com/darcys22/godbledger
Its heavily inspired by both ledger and beancount but my biggest issue with them is that text files arnt great for double entry bookkeeping. Having a relational database is the better option which is what GoDBLedger has.
After a certain point a business cant keep track of its transactions in text files because there are simply too many of them, so these systems really only scale to personal finance levels (few hundred transactions maybe thousand transactions).
In addition building plugins that can import your whole text file into a sql system so you can query them is redundant. Just have it in a relational database to start with.
If you store your transactions in a plaintext format, why don't you use CLI tools? I just renamed a bunch of accounts using sed for instance, in a hledger journal.
I actually love ledger, and GoDBLedger can read in ledger files for this reason. I hate to throw negative feedback at other open source systems because the ultimate goal is to automate the financial process and we are really all working together to achieve that.
The way it works now is that I generate a directory structure from my bank account transactions. In each directory I put a text file thay describes my accounting transactions. The program parses this and inserts it into the database. From there it can generate reports.
C´mon, you do not like textfiles but you do NOT support PostgreSQL?
Anyway, Martin is working on v3 which is moving the core of the parser into a complied language, in order to make it even faster when parsing large datasets.
I feel like I’m getting the best of both worlds :)
But I don't see why I'd be an issue to convert ledger to beancount.
I'm able to auto-categorize many of the transactions this way but there will always be some to which I need to manually apply categories. For example, entering "Expenses:Restaurants" for a restaurant I've never been to and thus don't have a corresponding matching rule. I also occasionally add new matching rules and then re-run the importer to generate a new ledger with the new auto-categorizations.
Once I edit the ledger file though I can't rerun the importer or else I'll lose my manually entered labels. Do I just add those one-off entries to the matching rules? Or is there a better way to handle this? Would be curious to hear how others are approaching auto-labeling expenses, etc.
I am intrigued by these ultra simple setups. However, most accountants -- the people who will ultimately help you once your business is sufficiently complex -- will use Quickbooks or some other established software.
What I found was that unless you are using the same back end, most of them were going to re-enter everything anyways as they want to double check. So it was more important to have the accounts & conventions they expected, rather than the actual software.
In my case, the data entry cost was about < 10% of my bill, so moving to the same software as them wouldn't be a big saving.
I've not used beancounter, so I'm not sure how much (if any) work would be needed to get the same support I had in gnucash. Mostly the complexities of invoicing and taxes in multiple countries and currencies, payroll, etc. One big issue potentially is you'll need to be able to produce proper financial statements for your jurisdiction (at least, countries I'm used to). Not sure if/how that is supported out of the box so might be some work.
- Aktiven <-> Assets - Passiven <-> Liabilities - Betrieblicher Ertrag <-> Income - Betrieblicher Aufwand <-> Expenses
I'm not sure I understand what you ask about rollup, but so far I haven't found a case I couldn't fit within Ledger. For VAT, I have a flat VAT rate that only applies to my turnover so it was simple to automate (reduce turnover by x% and put that amount into Owed VAT).
Keep in mind that these projects were born for personal accounting that use the cash-basis approach. Business are more complex and require an accrual approach, simplyfying we can say that they recognize revenues in a period only when these are certain, and recognize expenses related to these revenues in the same period, even if the expense occurs before or after the revenue.
You don't need to perform these steps for personal finance, you just "follow the money".
I'm enjoying plain text accounting for personal use but I struggle to see it applied in business use cases.
It's available in F-Droid.
I found it worked better for me.
But I only use that for cash or other things where I don't get a receipt. For the 99% I do get a receipt, I take a photo of said receipt. Then I deal with it later at my desktop or laptop where I have a real keyboard.
I actually wrote some helper script that works on top of Dired (in Emacs) to gather things like Payee and accounts using a completing-read (narrowing) style interface. This reduces friction A LOT, which I think is one of the keys to success in any chore.
Any recommendations for guides or short books to get into accounting for personal/individual finance as a total beginner?
Or maybe this? https://cone.tangential.info/
Especially due to the error-prone-ness and slowness of using mobile screen keyboards, you need guided/assisted data entry; a number pad only when you need to enter numbers or dates. A "use today's date if blank" field, selectors for accounts and tags, etc.
I couple it with Syncthing, and Fava running on a different computer served over the local network.
I would like a phone app to run on the beancount file, but the only apps that purport to do this have <1k downloads and no ratings, so I'm not going to trust them with my financial data
Parent-Teacher Association??
I thought the photos app was a fairly innocuous form of screen time, not realizing that it was so easy to make a purchase! No password confirmation or anything.
Fortunately support made a one-time exception and refunded the payment even though it was well past the cancellation window.
I still removed all the payment methods and addresses from our Google Pay account because I couldn't figure out any other way to guarantee that this doesn't happen again.
Frictionless payment flows do not mix well with toddlers.
Though is the root issue just communication? At some point transactions need:
1. Recorded
2. Categorized (optional, but desirable)
3. Reconciled (This can only happen as often as bank statements are issued)
With multiple spenders, communication (to answer the question "what is this transaction for?!?!") is paramount and fits naturally at the reconciliation step or more frequently if desired.
The workflow described is synchronous, centralized and simple
The entire thread was about accounting, then a sub-thread on family accounting, then you pop in to just say 'I do my accounting solo, as does my wife" which may be on-topic, but just not relevant.
But the intention of your parent was "when funds start to co-mingle or you get more complicated accounts" and saying "Don't co-mingle" doesn't help answer the question for those who do co-mingle and asked for help in this thread.
>The short answer is that my wife and I have different opinions about how to track data.
It supports preprocessing the csvs if you need to clean the data or compute some new fields which is really powerful. Once you are up and running it only needs some minor updates each month to map unidentifiable transactions I can do this in under an hour these days.
You can build rules too so the more you use it, the more automated it becomes.
From there I just use gmail filters to put all of those in a folder and then wrote a script to download, parse and enter those transactions on my ledger with “Expenses:FIXME”
Once a week or so (when I have around 10-20 transactions) I import them and go through them, asking my SO if there’s any transactions I don’t know of.
Alternatively I also just make one transactions to put in an account that is fully my SO’s and one that is mine (mostly for both of our hobbies) and those I don’t track, except for the monthly deposit to those
I have email notifications for all transactions and a maildir on my dev machine. I just need to write some code to make it all happen.
(If this is what you meant by family finances-- as others have pointed out, there's a difference between budgeting and accounting. YNAB has met all of my family's needs in the financial realm.)
I keep track of my expenses by CC and cash account, by the month. I don't care that I spent $5 on a coffee. I do care that for the month of January I stayed within my $X amount, and that for the year I'm on track for my savings goal(s), retirement, etc.
So I keep 2 spreadsheets, 1 is monthly, to ensure the bills get paid and the 2nd is my NW that tracks my retirement and savings goals.
I would advise not to worry about petty cash. Don't get pedantic. Worry about the big things. Just make a cash account for petty cash.
If you don't share accounts then just track your stuff and set up reimbursement account(s) for your family. You'll only know the net amount you need to transfer between yourselves to square up at the end of the month or whatever you agree to do.
It's not perfect but it's the best system I've tried, including YNAB, because I get paid biweekly and YNAB forces you to budget monthly.
Thanks.
In the beginning we--like virtually anyone else doing accounting--used QuickBooks. As we grew QuickBooks was less and less of a solution because of the issues we had with multi-user collaboration along with historical/rollback change tracking, e.g. knowing who made what change where and when. In other words, we started to outgrow QB. Further, there were lots of accounting needs we had where QB continued to fall short and get in the way.
We started to look around at various proprietary and open-source systems which were either locally installed or cloud hosted. Local installations came with problems surrounding collaboration and OS compatibility (Mac/Linux/Windows users) while cloud-hosted ones had issues around durability or longevity of the underlying company and sub-par data entry latency.
Further, we had additional needs including strong tracking of cost basis, alternate currencies, the concept of "lots" (FIFO/LIFO/cost averaging) of stock sales, short/long capital gain/loss, along with cost of goods sold, among other things. Finally, we wanted to get away from all paper records entirely meaning that any paper receipts would be scanned and tracked alongside the corresponding transaction which helped with collaboration because it made researching a transaction easier. (The receipt tracking we did via a customer document meta tag along with an associated ID attached to that tag residing in each transaction.)
As a software development company, we aren't afraid of a little bit of code and so the idea of a simple, plaintext accounting system with version control really appealed to us. The strongest contender that fit the model of our specific requirements was Beancount among all of the ledger-based derivatives. We've been very pleased with our results.
For our taxes, we do an HTML export of the financials using "bean-bake" and our CPA can navigate our financials without trouble. We use Fava (a Beancount web GUI) for various reports.
One place where Beancount really shines is the ability to edit multiple transactions at a time. Because it's just simple text files, you can easily use the "multiple cursors" option to make bulk edits or you can do simple search/replace. Because of Git-based checkpoints, there's no worry that we're going to break someone else while experimenting with an idea of how to express a given concept or the meaning of a set of transactions in the financial records.
That said, one place where Beancount still needs some polish is on the reporting side. Even with Fava, getting reports has never been as easy as QB. It often takes a little bit of Beancount SQL (SQL-like) querying to get the data you want. For getting loans and other things where paper-based reports are required this can require a bit of extra time.
Another weak point (at first) was for transaction import. This was soon overcome as we wrote some code that uses OFXTools to bring in all the credit card and banking transactions and to write them to the appropriate text files and then we go back through and categorize.
One last thing to mention is that, while the core business logic for Beancount is sound and far more capable than any other accounting program we have experienced, I can see how the choice of Python (or more specifically parsing of large text files with Python) can start to be problematic in terms of performance and delay. We're seeing the slowdown a little but it's not terribly inconvenient and doesn't interrupt the workflow. It's more akin to the compile time for a small application, e.g. 2+ seconds. I could see file-based checksum/caching into an internal binary/proprietary format helping quite a bit--kind of like those annoying little python .pyc files that show up when you execute Python.