Plain Text Accounting, a guide to Ledger and friends
plaintextaccounting.org
plaintextaccounting.org
(1) One of the hard problems in personal finance is data capture and input validation. Programs like QuickBooks and Xero don't just hide the GL because they're being mean, they're doing it because even experienced bookkeepers make mistakes on the general ledger all the time. "Since it's a text file it's easy to read" solves exactly the wrong problem: you want it to be easy to write, and easy to write CORRECTLY.
(2) They're the sort of solution that seems fantastic if you don't actually operate at scale. But once you start dealing with tens of transactions per day it simply is not sustainable.
Safe to say: I'm not really a fan.
With hledger and vim, I typically search and copy a previous similar transaction, then search and copy similar items. I have a keybinding to change the date, so like 3\d changes the date of the transaction I copied to 3 days ago. Overall, being proficient in vim, capture takes very few keystrokes.
Since the files are plain text, I can easily e.g. list all products I've ever bought with a `grep | sort | uniq` pipeline, to check stuff like whether I put a product identifier under a wrong account or used different identifiers for the same product.
Also, since the files are plain text, I can track changes with git. That gives me clarity over changes in the sense that I can make a large change (e.g. reorganizing accounts) over historical transactions and review the changes before committing.
Regarding large changes, which I think is also a type of capture, I can easily make them with `sed -i ...`, vim macros, awk, etc.
> and input validation
After I've captured, running hb (which is a personal alias for `hledger balance`) results in hledger raising an error if a transaction is not balanced. I can also add an assertion `= X` to the end of account changes to assert what the account is supposed to hold after the change. The number comes from physically checking the account, whether that's by checking the bank app or opening my wallet and counting. hledger will raise an error when an assertion is wrong.
> Programs like QuickBooks and Xero
Being unfamiliar with QuickBooks and Xero, how would these programs do capture and validation better for me?
It also memorises what transactions go into what category, so after you categorise, say "PURCHASE LIDL" as Expenses.Groceries, it will automatically match future transactions with the same name to groceries as well.
Most of the time, there's no need to remember. Just put the receipt in your pocket, and when you get home you can further postpone it by stacking it with other receipts on a receipt spike. It's only the cash transactions that are best not left postponed for longer than a day or 2, but if you just scribble down the important bits on a scrap of paper, you can postpone it like any other receipt.
I usually just hold onto the receipt for whatever I do until the end of the working day and get it entered that night. But if a receipt or my pay app isn't an option then I can enter the transaction right there on the phone before I forget.
Emacs users should look into Flycheck integration, eg https://hledger.org/editors.html#flycheck-hledger . Real time indication of parse and balance errors as you edit is nice!
Also I use a credit card for most of my expenses, and there are only a handful of bank transfers per month from my access account. Every first day of the month I pull out last month's credit card statement and log all the transactions. It usually takes less than 30 minutes.
[1]: https://github.com/roylez/dotfiles/blob/master/misc/bin/la
Now I just use a simple discord bot to capture, which is synced to a private git repository
Messages look like
&out soba shop
Soba 800
Beer 700
Or &p #custom:category payee
Item (int cost)
Item (int cost)It's also true for business accounting. Getting data into your accounting/ERP software is always a clusterfuck, and I've yet to see a process that didn't require many hours/days of reconciliation and posting manual journal entries to close a reporting period.
It's in the customers' best interest to have a universal format.
It's in no one else's (read: especially vendors) interest for there to be a universal format.
Which is why you generally only see it when it's mandated by a coordinating party in the middle (e.g. Medicare).
In the US, open banking would probably look more like "the IRS requires a bank to have accounts readable using these formats, or otherwise be able to provide same manually, when requested" to prod industry to do it.
The challenge is that no two businesses are really alike, so it's difficult to create a standard that can capture all the nuance of running a business. GAAP/IFRS-compliant accounting requires double-entry accounting, so many ERP systems share a lot of the same things (account, debit/credit, accruals, receipt of goods, month-end close) that are needed to spit out the three sheets and other artifacts for financial reporting.
A running joke in the industry is that you don't customize your ERP for your business, but customize your business for your ERP. You end up with a lot of "middleware" that essentially adapts your operational systems into the format needed by the ERP to do its job, and your staff will design workflow processes specifically centered around how the ERP works for things like closing the books or managing inventory.
It can't tell you how to structure sales to account for taxes across multiple countries, or how to accrue for gift cards/liabilities, or recognizing revenue from annual subscriptions, or deprecating assets, or amortizing expenses. Each ERP and business will do this in slightly different ways.
But what it can tell you is how to represent those results when everything else is said-and-done.
But back to reality, everything in life is time-consuming. But, that is why it is so important to have a healthy profit margin and enough revenue in to hire a bookkeeper - so it takes you zero time.
But, even still, the real issue is that most business people don't know sh-t about accounting, don't know about accounting controls, don't know about so much, that they can't really evaluate anything anyways. How many companies have I been with or known someone that works at them that have been ripped off by embezzlement, because the company didn't know about controls. Too many. Rita Crundwell who was the Comptroller and Treasurer of the town of Dixon, Illinois is an especially juicy case of embezzlement. Crundwell stole nearly $2.5 million per year from the city. Crundwell with embezzling $53 million over the prior 22 years. And I think it was more, because she worked for a longer period than that, but no records were available. There was a great analysis of it, and wow, was it great. Just so fascinating.
I have a lot of experience in accounting and bookkeeping, so I really enjoy it. I don't know, it doesn't really take me a long time, I do it every day, so it takes me less than 4 or 5 minutes per day. Sometimes a little longer, sometimes less.
Which mistake do you think is more common with PTA than with other programs?
2. What exactly would not be sustainable with PTA which would be with other programs?
This is why I've started looking into tools which automate the data collection. In the UK (and EU more broadly), after Open Banking got introduced there are many apps popping out which basically connect to your accounts (Credit Cards, bank accounts, etc.) and combine all data into a single place. Does Plaid not enable this in the US?
It's not "100% accurate" but most enable you to do auto-categorisation, smart reporting and budgeting as well. They are not open-source but do much more than I'd ever be able to do with PTA (not because it's not supported but because it requires effort). Of the 2 bad solutions, they seem to be the less bad for me.
Some example apps: Emma, Yolt, Lumio.
The reporting and analytics functionality of the ledger-likes is AFAICT unrivaled, and is the REAL reason to use them (not the ostensibly easy-to-use “plain text” general ledger aspect, about which I agree with you)
Record significant spendings, and account the rest as "other" - it should be the amount allowing to balance the ledger, given the current assets, income, and the significant spendings.
My problem with ledger was the absence of correct support for multi currency accounting.
(I wrote a text based ledger tool myself, and then switched to storing data in PostgreSQL)
What is the GL?
The most annoying part of the process is being required to download CSV files from each of my bank/cc/investment accounts.
I thought about building some APIs to be able to download this data automatically but some of my banks do not have publicly available APIs. That's hilarious to me because something like Mint or Plaid can get API access to my bank transactions but I, the owner the of the accounts, cannot.
Not only that, but you can only download CSVs for the last 2 years. Mint/Plaid can get the entire account history.
EDIT: It seems Plaid has negotiated API access with some institutions at this point, but historically has primarily relied on screen scraping.
Unfortunately, only one of my banks provides OFX access at all (for a monthly fee), and I learned I’m not disciplined enough to keep up with the other banks manually.
Its annoying because I’m pretty sure the other banks do have OFX services somewhere, they just dont provide it to all customers, plus paying a monthly fee for the one bank that does just feels exploitative.
A less reliable, but more accessible solution might be to parse emailed bank statements, but those only go out once a month, whereas OFX can show new transactions within minutes/seconds.
Unfortunately, it's far from true that all institutions supporting Quicken downloads can be found via ofxhome. That list was originally sourced from parsing MS Money API, and has been only sporadically updated in a manual case-by-case fashion. Also ofxhome is set to go away pretty soon.
But we do what we can
Plain Text Accounting - https://news.ycombinator.com/item?id=25745615 - Jan 2021 (200 comments)
Plain Text Accounting, a Guide to Ledger and Friends - https://news.ycombinator.com/item?id=11164330 - Feb 2016 (26 comments)
Edit: I've changed the submitted URL of the current thread from https://plaintextaccounting.org/#plain-text-accounting-apps to the home page, which gives more information. See also https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...
They shouldn't do themselves down. I am an accountant and I have used several large systems that do exactly this in the database. It is also by far the preferred way to export a trial balance or gl account from any system to excel. It is the norm in the industry and in no way an innovation of plain text accounting.
- 2003: John Wiegley creates the field with the Ledger command line tool and its file format.
- 2007, 2008: I and Martin Blais create two new "ledgerlikes", hledger and Beancount, with their own technologies/priorities/maintenance styles.
- 2003-present: Community grows; many helper scripts, tools, docs, and other ledgerlikes appear.
- 2016: To grow the ecosystem, organise resources, and reduce support effort, I coin the Plain Text Accounting phrase and set up #plaintextaccounting IRC channel and https://plaintextaccounting.org. (And in all honesty, also to encourage objective comparison shopping and regain a little oxygen for Ledger alternatives).
- 2017: Colin Dean sets up the plaintextaccounting subreddit.
- 2021: The community continues to grow. The IRC chat is bridged with a Matrix room. Ledger has been "done" for many years but may have a new release soon. hledger has been steadily developing, with quarterly releases. Beancount is in process of a big rewrite. There are some other strong ledgerlikes, currently with much smaller userbases.
Thanks a lot for all of your recommendations !
My beancount file is nearly 3 years old now and I love it. Maybe it's overkill for personal finances, but I've tried countless other PF tracking tools and online services and none of them have the flexibility you get from tools like this. For example, tracking RSU vests, my house purchase etc.
Combining beancount with Fava gives you a nice visualisation tool as well which auto updates when it notices the underlying beancount file has changed.
I edit the file in Emacs using the beancount-mode, it's actually the only thing I use Emacs for but I have a bunch of custom elisp functions that add things to the file whilst I'm editing it.
I don't bother with any auto txn importers, which does make it a manual process but it's like 10-15 minutes a week at most. Additionally most of my purchases are generally quite rote so a lot of them are just copy/paste from previous txns and modify the date and amount. This might not scale if I have a partner, but for now just Emacs + vim keybindings + a few custom elisp fns + a little patience works for me!
Recently added budgeting to the mobile app. Most of the user-facing improvements have been cosmetic to UI design, colors, and phrasing.
The reports still are not at feature parity with the old YNAB4. Fortunately there’s an open-source Chrome extension - Toolkit for YNAB that improves reporting and charts. However, I find it disappointing this functionality isn’t built into the App.
Auto-importing isn't really a point for me, since YNAB has always only supported American banks.
I think I'll reinstall old YNAB these days.
Stuff like that is not as simple if the backend is an SQL DB.
> I have not used the SQL output.
I've used it to process the register output in a pipeline.
Sometimes, my `hledger reg` queries result in multiple quantities on some transactions. The terminal output leaves the columns of the date, transaction description and account blank, for pretty-printing purposes, I suppose. Those blanks complicate adding up the quantities per transaction using a tool like awk. There's also the JSON output format, but the depth of the objects around quantities makes the CSV still simpler to process, even though I need to process the quantities' text in SQL so they can behave like quantities.
Just sharing how one user uses hledger.
Every time I dig into accounting and try to learn it more in depth, it seems that the foundation is trivial. And it coasts on a few "clever" implications from the double book assertion (income/expense accounts), almost in spite of how they don't make sense. You can't classify an item as multiple expense categories without growing the semantic model, so why do you even need the concept of an expense "account" in the first place, as opposed to a single entry in your cash account, tagged with categories?
Meanwhile the bulk of the real complexity revolves around tax rules, which always seem to get left out of "accounting" proper, even though they're the overwhelming reason people to go accountants. I actually installed QuickBooks the other week to see what the gold standard looked like, and it ultimately just seemed to be another way to keep basic credit/debit accounts, with little in the way of preparing/optimizing taxes. Its main value add seemed to be in generating paperwork (eg invoices), that while perhaps nice to formalize, would just increase my scope rather than solve my problems. I expected that it would have automatically prodded me into creating a depreciation schedule and things like that, but it appeared I had to set things like that up manually in terms of basic accounts?
I'm asking because I see people talk about accounting as if they're getting a lot of value out of it. But every time I've dug into it thinking there must be something I'm missing, I've never ended up seeing that value.
I guess one could write tools to ease importing data from banks, brokers and the like.
Currently I don’t do the full accounting stuff. I just keep track of my net worth using an Apple provided Numbers sheet. It’s easy and quick, just occasionally logging into all my accounts and then updating the values in the sheet. This way I don’t get the historical overview, but at least the data can be updated in just a couple of minutes. As long as I see my net worth rise over time, I am happy.
—-
Unless you do dozens of transactions every day, it's not much of a hassle to keep track of them manually. If you're like a day trader or a small business owner then yeah, you'll probably need some helper tools.
I don't have too many transactions indeed.
I am actually contemplating to switching from my Numbers sheet to the Ledger command line tool. I do like that this tool is multiplatform and based on simple text files. And it shouldn't be too hard to write a few scripts to convert data from my banks and brokers into the format used by Ledger. I guess I'd need a day to setup everything, so perhaps it'd be a decent weekend project.
Eventually I would like to leave the macOS platform and not relying on Numbers would make the switch just a little bit easier.
In my case, I grew frustrated with my approximations. Computing the difference between my balances at different points in time hides the details that would make it actionable, or give any insight about past choices. So in practice I would go through my past transactions, try to get an intuition of what happened. At this point, switching to proper accounting is just about scratching an itch and making this whole process more efficient.
My primary consumption method is hledger-ui, a nice curses-based UI. It's got a `watch` mode so will update in real-time as you make edits to the source database, which is nice.
The ONE thing I miss from the old QuickBooks UI world: Entering categories for transactions directly in my text file is tiresome. QuickBooks had a super nice auto-complete feature wherein it would remember the category as you were typing a transaction, which made data entry a snap.
Anyone have any suggestions on how to replicate this inside a text editor? Or any other good way of quickly doing transaction entry?
hledger add, hledger-iadd and hledger-web are some of the data entry uis with auto completion.
Alternatives: other ways of generating entries means less manual data entry, such as copy/paste in your editor, predefined templates (org-capture, yasnippet or equivalents), or conversion from bank csv data.
[PS I realize now you were asking for more than account name auto-completion, ie: filling in defaults for all postings (legs) when you reuse a transaction description. I don't know of many PTA tools doing this yet; hledger add is one.]
For Normal People:
Double Entry even though may sound simple, it is quite a difficult concept to grasp. For the sake of double entry we use counter entries that take months to grasp. For example - Amortization, Allowance etc.
More over double entry method strives for standardization of entries. Having a plaintext non-autocompleting solution without accounting for only handful fixed type transactions....I just could wrap my head around it.
Moreover with online connected mobile solutions that are most of the time free why bother using this when the steep learning curve doesn't yield any marginal benefit.
For Accountant:
Excel is god and whatever accounting software the org is using is an acceptable solution
---
Even though the FAQs and the previous discussion has addressed this exact same issue. I really don't know what type of people would use plain text command line solutions except for the sake of using plain text command line solutions.
PTA tools, some of them at least, can scale well from beginner-level single entry accounting, or even zero entry "accounting" (just a list of dated event descriptions), to fully general double entry accounting that can model any financial event. I always encourage people to start out simple. What usually happens is they learn and become more interested and more sophisticated as time goes on.
So I would say one group of people this appeals to is people who want to get their hands dirty with Double Entry Bookkeeping, with the support of a lively and enthusiastic community, in order to understand it better.
Another is people who are expert with text editors, scripting and programming and want an accounting system where they can bring these skills and insights to bear.
Another is people who want total control and privacy of their financial records.
Another is people who want their financial data to be accessible and usable for a very long time (even without software, if it came to that).
Even for using of-the-shelf software from a third party, you need to learn the idea and skills behind double entry bookkeeping. The software is not going to fix that for you.
P.S. If anybody is interested in a simple locally deployed PHP-script to do bookkeeping, send me a message. There's also a simple timetracker, a balance sheet, profit/loss statement, etc. You need a local server and MySQL database to run it. I guess in it's essence it's applicable worldwide, but if you're from The Netherlands, it's ready to do taxes for you.
Granted using Emacs isn't exactly for everyone, but there are a bunch of tools out there to make editing the text file much easier
I just send CSV exports and reports to my accountant.
I'm an independent software developer working on my own. I guess it's useful for people like me.
It’s not particularly sophisticated but it works and it’s simple enough to live with.
- For example, instead of someone needing to remember passwords to five different current accounts and credit card sites, there are ruby scripts that download the data, convert them into journal entries and get committed to git. (Commercial packages can read your credit card data too, but there will always be a data source you need that they don't support)
- The scripts get committed and version controlled, but the ledger files themselves go into git as well. So you can bi-sect, revert, merge. All the tools git gives you that are an order more powerful than what an accounting package will have.
- As an organisation grows it will run into weird business specific processes, e.g. some tax or reporting requirement that your off-the-shelf package cannot do. If it's all just plain text files and code then the reporting requirement is one more ruby script away. (I've been a consultant in a previous life and you wouldn't believe the man hours that get lost in accounting departments with people doing these manual chores)
- Filling out quarterly tax statement is again just a single script. Took a bit of time to get it to work but now saves a lot of man hours each time.
There is a disadvantage: your average accountant won't know what ledger is or how to use it. But we have yet another little script that converts the ledger output to Excel for use in audits, etc.
I forgot to mention I studied accounting in uni and after investing/wasting my time into understanding CLI and terminal application I have come to the conclusion that they are simply not for me. Specially if you are not using Linux in all your applications and even that is another issue. I digress...
I am just incredibly intrigued by your organization's practice. Who are the people that are interacting with accounting system? I know it requires a more than capable IT person to feel comfortable using a CLI. So, do you not have assistants or accounts professionals whose main job or one of their main is to do accounting? Or is it all IT or programmers that are interacting with this system.
How large is your organization. That is a question also.
How long did the organization ran for to settle on understanding the basic requirements from the accounting software to commit to creating a CLI solution?
How are you outputting reports that are well formatted?
Audit usually requires a walkthrough process of accounting. I couldn't imagine any auditor not being surprised to see a CLI based software and let alone attempting to understand these.
I have thought about creating my own CLI solution that just appended to my workbook. But it is just not scalable and simple modularity requires going down the codebase which is not something I imagine doing on the weekends.
And that is why the barrage of questions. With CLI applications (not even custom GUI application) the layer of complexities for the sake of simplicity means that the person using it has a ton of highly specific needs that is not the standardized and they also have a very different definition of the word "simple".
That's small enough that all accounting can be handled by a single person, me, including updating the scripts that keep it all running. They do need regular upkeep as a result of changes in regulations, APIs, etc. It takes me about two or three days per quarter, and a few more days at year closing. There is a script that generates a pretty Excel workbook from the ledger output with a familiar looking dual ledger balance sheet and another one with a profit and loss statement, any accountant should be able to read those. There is not often a need to involve an actual accountant or do audits though, the company is privately held.
Teaching a programmer the basics of accounting is pretty easy. Would this work in an organisation that's not a software company where you would need to teach an accountant scripting, git, CLI, etc? I don't know.
Over the years, I’ve kept extending it to calculate and generate invoices for my clients and email it to them with a payment link automatically. Also combined with gnuplot its a very minimal and excellent tool for visualising my finances in any way I want to. My plan is overtime to combine it with streamlit.io to make interact-able projection tools for myself to tinker with my finances.
And its community is super fun too ! Ledger and Hledger are some of my favourite tools.
Also can someone link to what the actual text file looks like? The format. I can't find a decent example, I just want to see this human readable format, thanks.
An example of Ledger/hledger's main file format:
2021-01-01 opening balances on january 1st
assets:checking $1000 ; a posting, increasing assets:checking's balance by $1000
assets:cash $100
liabilities $0
equity $-1100 ; each transaction must sum to zero
2021-03-05 client payment
assets:checking $2000
revenues:consulting $-2000 ; revenues/liabilities/equity normally appear negative
2021-03-20 Sprouts
expenses:food:groceries $100
assets:cash $40
assets:checking ; a missing amount will be inferred ($-140 here) 2021-03-20 Sprouts
expenses:food:groceries $100
assets:cash $40
assets:checking
This means you took 140 from your checking account, withdrew 40 that you now have in cash, and spent 100 in groceries. Does that still make no sense?Even though you've explained it I'm still struggling to make sense of it. The 3 indented lines don't related at all in an intuitive manner at least for me.
What they should be showing beginners is:
2021-03-20 Sprouts
expenses:food:groceries $100
assets:cash $40
assets:checking $-140
How’s that?If you're interested, this seems like a fairly decent guide: https://www.open.edu/openlearn/money-business/introduction-b...
Where it goes and comes from are "accounts", which may or may not correlate to actual accounts like you might think of in personal finance. There are 3 categories of accounts: expenses, assets, liabilities. You own the second two, they are your cash, checking, savings, brokerage as assets and your mortgage, car loan, CCs as liabilities. Expenses are where you spend money outside of your own accounts.
Before getting to expenses, you may have accounts like this for your assets and liabilities:
assets:cash $100
assets:checking $2000
liabilities:cc $-100
liabilities:car $-10000
Whenever you transact your money goes from one place to another, so a car payment may look like: 2021-09-05 Car Payment
assets:checking $-500
liabilities:car $500
Positive means that that account gets the money, negative it loses the money. Note that it sums to $0. If there were an interest component we may do this: 2021-09-05 Car Payment
assets:checking $-500
liabilities:car $400
expenses:interest $100
Expenses are not owned by you. They can be broad categories or more specific. Maybe you have two residences and utilities at both: expenses:primary:electric:edison:1234 -- your account number
expenses:secondary:electric:pandg:1234
I wouldn't go that far, I might do: expenses:house:electric:primary
expenses:house:electric:secondary
But the choice is up to you, both work. Think of them as hierarchical tags.Additionally spreadsheets group logic and data - whilst this is very flexible csn get hairy if you are not disciplined.
Beancount files are just text, and the logic part comes from the beancount engine so there is a clean separation.
My system is obviously not fully tracking everything / double entry. It's just a log transactions on an account. I have a sheet for each account.
Anyone have experience with the apps mentioned in the site?
It does have some limitations though, as cone is append only, it cannot edit transactions. For this, I've had to fire up vim a few times in Termux.
I have also created a few scripts using ledger, gnuplot and termux-open to generate graphical reports, and have exposed these via termux widgets. This gives me a one-click access to real time reports on my phone.
PS a little daily-ish data-entry-and-reconciling ritual, with a pleasant tool setup, can be quite satisfying! I actually look forward to it. (That right there is part of my PTA success story. I used to suffer a lot of stress around all things finance.)
What libraries were used to do that? I don’t see a package.json
Lemme know if you ever need any help with ledg!
* Use a version control system - keep journal files in eg a github repo; control access, manage pull requests, audit changes in the usual way.
* Use multiple files - the journal can be composed from multiple files, each with different filesystem permissions/accessibility.
* Maintain files on a server, and run a web UI like hledger-web or fava, perhaps behind a login form, to give read-only or add-only access to remote users.
Also, I cannot and will not share personal financial data to yet another 3rd party SaaS. So please refrain from mentioning YNAB and the likes ^^
balancednoautoconversion validation may be relevant, if so (https://hledger.org/hledger.html#strict-checks)
Leger printed nonsence for such roports.
I used to pay taxes in multi currency environment, and know the official rules here, in Belarus, how to calculate the indicators when converting to a single currency.
Also I once came across US guideline for that - the same simple principle.
So I guess that's a very standard aporoach in accounting.
Ledger does something strange instead.
I remember also some large blog post was circulating, where the author "explains" how to do accounting in two currencies, in some overcomplicated way, that only works for two currencies. This link was usualy shared between people as an ansver to the multi-currency question.
All the major PTA tools support this - perhaps the chat could help ?
> my net worth, my income over period of time, capital gains / losses.
The first two are easy, the third is more complicated but also possible - unrealized gains at least, I forget how easy realized gains are in the various tools, it might not be fully automatic.
If you can, lets chat, thank you.