Hacker News Reader

Comment by wasdfff | Hacker News Reader

Do banks have anything you can use to automate getting data off their websites? My annoyances lie in generating the csv files. My checking account outputs an arbitrary range of dates, however many transactions fit on pg 1, while my credit card from the same bank outputs csvs by billing cycle. It makes it a little annoying going into the bank website and generating these files across relevant date ranges.

View on HN

In my experience (USA), most banks will give you the option of downloading a CSV for the last two years of transactions. Beyond that, it’s a challenge.

Some investment brokers will only give you PDFs with tables in them. But there are some pretty amazing libraries for converting these into CSVs, that have worked well for me.

Some people use plaid, but I didn’t like the idea of introducing a middleman-dependency.

What I’ve been doing over time is downloading all of the raw transaction history I can get, and storing it in a git repo. Each month when I “top it off”, I just use git add -p to add only the latest bits. The most annoying part is when they change their CSV format on you (Chase did this recently) and you need to reformat all of your historical data to match.

Maybe your bank is better about this and offers OFX, but ultimately I gave up on this approach due to my bank adding more aggressive anti-scraping measures over time. The credit card sites were even worse about this.

Now I have a Postfix server setup which accepts the daily transaction summaries they send and makes CSVs out of those. There's still some HTML munging involved but I've found it much easier to let the data come to me than to go out and try to fetch it myself. In my experience financial institutions simply don't have a lot of reliable automation options accessible to the normal consumer.

> Do banks have anything you can use to automate getting data off their websites?

There's an Open Financial Exchange (OFX) standard that's supposed to be for this, I think. I came across it when someone mentioned the ledger-autosync [1] tool that tries to use it to get your data from your bank automatically. If I understand correctly (no guarantee on that), this automation would work for institutions listed here [2].

[1] https://github.com/egh/ledger-autosync [2] http://www.ofxhome.com/index.php/home/directory/successfulRe...

I just got started using the ofxget command in the ofxtools package. It was a simple "pip install --user ofxtools". After that, I read some of the docs for ofxget and configured these with the settings from ofxhome.com and I've got automated ability to pull in my transactions for credit cards, checking accounts, and even long-term investment accounts. It's really cool to interact at this low level because there's so much control compared to QuickBooks/Quicken.

Reply on news.ycombinator.com