Hledger: Robust Plain Text Accounting
hledger.org
hledger.org
I feel combined with machine learning and storing hledger records in some large databse, it can become a powerful automated accounting system which SAP and large vendors can just dream of.
Works quite well
Our company uses a plethora of (mostly) ruby scripts to get data from banks and the billing system API and a few other sources like exchange rates websites and converts the output into ledger files. And then another bunch of scripts takes the output from ledger and sends it to the API for, for example, the government VAT payments. (Our company is by no means big, but multinational with business in 7 currencies).
There are a number of test and assert statements along the line to do sanity checks with a manual review since sending out incorrect values could potentially be expensive / illegal. It works great. The whole thing is stored in git so any changes are auditable. We need some of the power features of ledger though, so I doubt ledger will work for us, but I'll give it a spin.
Distant future is hard to predict but here I go: hledger or any good parts of its design are still around and still providing value. Any past data kept with hledger is still readable and easy to convert, with or without a working hledger. Highly polished and efficient UIs are available everywhere they're useful. Data import and multimachine syncing are routine and just work. Data hygiene/security and personal privacy are routine and just work. Performance is high, "big data" is no problem. Resource usage is low, these tools continue to work in the turbulent and resource-constrained future. There is a strong global community of financially literate, capable, empowered, prospering individuals and organisations using efficient high quality universally available accounting tools to promote, enforce and create widespread transparency and societal insight in finance, business, politics and every ecological issue.
You can also fork and modify it to generate hledger format files directly.
I guess it's the most obvious question in the world, but: what are the big reasons I should switch to hledger? What do you see as the primary advantages of hledger over ledger?
Here are some things I've tried to do differently with hledger compared to Ledger. I feel we've had some success with all of these:
Project:
- be actively maintained and contributed to
- build out Ledger's concepts a bit more selectively/thoroughly/to a higher level of polish, taking advantage of hindsight, a fresh codebase, a well-suited language
Docs & design:
- all behaviour fully specified and documented
- all docs up to date and accurate
- practice documentation-first design and development
- more intuitive, learnable, consistent concepts and UX
- be usable for less-technical users as well as techies
- be easy to get started with as well as efficient for regular use
- explicitly try to model accounting concepts and help meet real world accounting needs
Software:
- be robust and dependable, with few user-visible bugs
- be easy to install and run well on all major platforms
- provide APIs/libs enabling more plain text accounting hacking among Haskellers
- be readable, clear, and a useful source of algorithm & architecture ideas for other implementors
- remain maintainable and cost effective over the long term
- achieve a long lifespan and provide a safe long term format for accounting data
Technology:
- clarify whether Haskell is good for real-world apps (yes!)
- learn more Haskell by building a real-world app (yes!)
- attract contributors by being a Haskell project (yes)
Out of time for now, I hope it's not too hand-wavy..
Not wanting to come across as too argumentative :). I'd prefer if there had to be bugs that they were user visible.
1. What was the motivation to create this instead of extending ledger?
2. How to track time?
That site is also worth checking out for more hledger materials, such as comparisons, talks, slideshows, and blog posts. Eg Andreas Pauley's quick intro slides (https://pauley.org.za/functional-finance-hledger) or my slow "Hands-on with hedger" talk and FLOSS Weekly appearance.
For 20 years, I have been using a format that combines a diary and ledger. The 'to' and 'from' accounts are either bank/asset accounts (sav, cc, hl for home loan, ...) or spend categories (snack, shopping, clothes, ... ). Each year I finish 31 December with a `# Closing balances` section for each bank/investment/pension account, then start a new text file `2019.txt` with an `# Opening balances` section before 1 January's entry.
Here is a typical day's format:
# 2019-05-27 Mon
2.50 sav>lunch Cafe Nero - Coffee
100.00 sav>cc Transfer - Credit card repayment
amt from>to forex: shop - desc [txn #]
# Diary
* <topic> - <what>
* Times - Sleep 01:00-06:00 5h. Run 45m. Work 08:00-18:00 10h.
* Work - Meeting with xxx on xxx. We agreed to xxx.
* Friends - Lunch with xxx
* Note - Running shoes are Asics GT-2000, size 48, model T500N.
# Links
* Hledger: Robust plain text accounting - https://news.ycombinator.com/item?id=20012499
When I got married, all our accounts switched to joint ones. I didn't want to track my my partner's side of our spending. So these logs became more a personal diary than a fully reconciled ledger. Anything I want to remember long-term goes either there or secured in a password safe.
I actually think personal logs are useful for all things, not only money. In my business we use a text-file with logs for all the events, money, business stuff that happens, and then there's a program that reads all the lines, parses them and turns them into a final "state" of the company with everything that is being done and so on.
From that we generate dashboards, reports and stuff.
This is the system we use: https://github.com/fiatjaf/journalstate (but really the power is in the concept, not the implementation).
I've also turned that into a TodoMVC app: https://journalstate-todomvc.alhur.es/, so you can see it in action in a fancy, useless way.
1/1
(food) 5
1/2
(car) 15
1/1,food,5
1/2,car,15
An advantage of something like hledger over a home-grown script is more robust and versatile reporting.
You can do the same as the scripts I wrote are specific to banks I am using and the pdf files they send, except one of the banks which provide qif format file which I convert to CSV. You can look at https://github.com/beancount/beancount/tree/master/examples/... for beancount custom import example.
They treat the statements as the single source of truth, with preprocessing and rules that can be used to turn them into hledger journals.
Also, running "make" prints a daunting 96 lines of text of which the last 50 (the ones that you get to see) are mostly useless. If I run "make build" it starts to download stuff from the internet, which I find extremely impolite, as I already downloaded the source code. What is the point of not including all the neeedded source code in the source distribution?
I've generally come to expect that any software I download and try to build might, through a package manager, pull in additional dependencies (rust, Haskell, scala/Java/closure,...). The only real outliers here seem to be C and C++, of the ones I've worked with.
A mature Haskell project such as hledger has probably a few hundred of transitive dependencies, so you won't be able to install them individually by hand. You might try to figure whether each of them is packaged by your distribution (unlikely, unless hledger itself is packaged), but that's not a typical path — just don't use make in that case.
Pip installing required packages is alright. But I expect setup.py to NOT call pip install by itself.
> A mature Haskell project such as hledger has probably a few hundred of transitive dependencies
That's horrifying!
"stack install hledger" would compile just the command line tool (not the curses & web tools), so less downloading.
For end-users who want minimum downloading/building, we recommend one of the binary packages on the download page (nix, docker, homebrew, windows/wine), but that's not you I think.
PS: "make" prints its help, a summary of all targets. I agree it's more than a screenful and will need to be viewed with a pager (I'm used to viewing things in an emacs shell). The Makefile is assumed to be used mainly by contributors and does a lot. What would you recommend "make" to do ?
I'm a GnuCash user, but my biggest gripe is not being able to encrypt my accounting file. I'd love to have it sync across my devices via x cloud platform, but I'm always paranoid about uploading anything that isn't password protected. If I was able to do that it'd be the push that gets me to switch from the GnuCash GUI to a plaintext account.
Maybe there's some other way I haven't thought of without a high barrier to entry like constantly mounting and unmounting an encrypted container to make changes.
I learned rudimentary double-entry accounting in high school, and I love that ledger enforces it so that I can be strict about my tracking. I still have to figure out how to produce balance sheets and income statements so that I can confirm that my accounts are balanced correctly. Honestly wish I had done this sooner.
Edit: Looking at beancount again, I should just use that instead of ledger. Ledger's simple, elegant, and nice, but I'm a Python guy, and I should just use what I know.
The hardest part is just getting started and getting into the habit of maintaining the ledger. Once you start tracking it's difficult to stop! Seeing all of your accounts line up is a reward in itself.
Maybe one of the other packaging methods will give you a more stable installation ? https://hledger.org/download
Like my parent commenter, I recommend just doing everything through stack. stack-static from the AUR [0] is the convenient way to do this (rather than dragging in a few gigs of ghc dependencies for stack only to have stack create its own local ghc installation as well).
Fortunately there's https://aur.archlinux.org/packages/pandoc-bin/
https://github.com/ledger/ledger
I used Ledger once upon a time, quite successfully! Worth a look.
Off topic, but as a student of Haskell for many years, I found reading (this morning) some of the code in the ‘tools’ directory of the github repo to be useful for picking up a few useful idioms.
So back to plain ledger. If I was starting over I'd probably consider beancount. But there's only so much I can do a day.
That works fine for asset type accounts. But when we are taking about as-was accounting it doesn't work well enough. Beancount has some interesting ideas about lot tracking but switching cost is too high - I have years and 35k lines.
- "The developers decided not to implement lot-based tracking" seems a misunderstanding. Perhaps "the developers have dragged their heels on exactly replicating Ledger's handling of lots" is more accurate (and I could justify that!).
- we have never had sophisticated valuation/investment features, but there's a bunch of work in progress on this area right now; more flexible valuation (https://github.com/simonmichael/hledger/issues/329) is a current focus and needs testers..
And, right now it should be possible to manage lots manually, similar to http://rantsideasstuff.com/posts/2018/07/01-tracking-investm... . It's a manual process in Ledger too after all. Let's chat in #hledger if you'd like.
If the point is to forbid you to modify even when you're the only person working on it, any common database will allow you to update and delete when you enter with administrative privileges, so I don't see the difference.
To check that the first N transactions in a file weren’t tampered with, you have to either compare each transaction with a backup, or hash them and compare the result with the hash you got earlier.
If, as I suspect, this ledger is append-only, both are fairly simple. If the ledger, say, had 42 lines earlier, do either
head -42 ledger | diff ledger.copy
head -42 ledger | my-super-hash
Also, some OS-es (e.g. Mac OS. See https://developer.apple.com/library/archive/documentation/Sy...) have the notion of append-only files. These, too do not safeguard against somebody rewriting the entire ledger, but they can make it harder to do that tampering.
Finally, I haven’t thought it through, but possibly, you can create a user on your system for your accountant, set the owner of the ledger to that user, and make it impossible for anybody but that user to delete or replace the file.
If that’s possible, it even would protect against reinstalling the system. You could recreate your accountant’s user ID, but not with the same password, so your accountant would still notice.
There's also non-envelope style budget reports built in. For more on these see https://hledger.org/manual.html#budget-report and https://hledger.org/Budgeting-and-forecasting.html.
Mine is written in TXR Lisp. It is not released to the public.
All input is recorded via function call expressions, which are of course S-exps. There is only one single macro in the whole thing: def-date-var defines a variable that exhibits different values depending on a effective date context (established by a dynamically scoped date variable). With this we can do things like different tax rates for different periods, without having to introduce such things as parameters appearing in every input item.
> The amounts within a transaction must always sum up to zero. As a convenience, one amount may be left blank; it will be inferred so as to balance the transaction.
I also have transactions that sum to zero. There can be three or more account deltas in a transaction, not only two.
This is not how accounting is taught in North America, though; I came up with it myself.
Inferring one amount is a bad idea. My system throws an exception if the transaction amounts do not add to zero.
However, the entry of items is done via various convenience functions, most of which generate the necessary transactions against all the right accounts. For instance, recording a new asset:
(record-asset 1 "Discombobulator" :cca-50 ;; ID, description, capital-cost-allowance class
(date 2017 1 23) ($ 79.99) ($ 4.00) ($ 5.60)) ;; base price, GST, PST taxes.
Different items need their own syntax. I don't want to record a simple purchase of a recurring expense in the same way that I record an invoice.
I bind a new variable when recording an invoice:
(defparml %inv-0042%
(new invoice
number 42
provider %my-addr%
client %foo-client-addr%
remit-to "Kazinator"
remit-days 25
date (date 2017 6 30)
items (list (new time-unit
start (date 2017 6 1)
end (date 2017 6 2)
hours 16
rate %rate-foo-client-2017%)
(new time-unit
start (date 2017 6 5)
end (date 2017 6 9)
hours 40
rate %rate-foo-client-2017%)
(new time-unit
start (date 2017 6 12)
end (date 2017 6 16)
hours 40
rate %rate-foo-client-2017%)
(new time-unit
start (date 2017 6 19)
end (date 2017 6 23)
hours 40
rate %rate-foo-client-2017%)
(new time-unit
start (date 2017 6 26)
end (date 2017 6 30)
hours 40
rate %rate-foo-client-2017%))))
This is then recorded using a plain function call to the convenience function record-invoice:
(record-invoice %inv-0042% "Invoice 0042 issued to Foo, Inc.")
When the invoice is paid:
(record-paid-invoice %inv-0042% "Invoice 0042 paid by Foo, Inc."
(date 2017 7 31))
To generate a nice HTML invoice, at the REPL, I just call the .(html ...) method of the invoice object, which takes a stream argument.
1> (with-stream (s (open-file "invoice.html" "w"))
%inv-0042%.(html s))