CLI tool for converting txn statements from Singaporean banks into a sane format | Hacker News Reader

CLI tool for converting txn statements from Singaporean banks into a sane format

github.com

79 pointshestefisk32 comments

Hello. I am an expat based in Singapore. I use GnuCash for managing my personal finances (it's a great tool). I was unable to find a way to easily import bank statements (CSV files with transactions) into GnuCash without all sorts of weird conversion errors. To streamline the process, I wrote a Perl-based tool for automating the conversion into something GnuCash can easily import. You might find this useful (or not!). Have fun!

I have about 15 Perl-based converters for various banks I use, from parsing Revolut's CSV exports to making sense of copying and pasting the statement from Citibank. Someone should really make a repo to collect these, although hopefully as more banks sign up to Salt Edge and friends it'll be less of an issue

Off topic - DBS is hilarious in Singapore, the "World's best digital bank" 4 years in a row (forget the years) - who did they pay-off for that title.

You can't even export bank transactions older than 6-months from your account in CSV. You have to download - individually mind you - pdf statements and then good luck. This is the worlds best digital bank?

Yeah banks here are so far behind what real neo-banks like Monzo are doing.

I left Monzo in the UK and came to Singapore to join Bigpay. We're building a proper digital challenger.

1.1m customers in Malaysia and about 1/10th that in Singapore.

My main pet peeve with Monzo is that it is App only which can be quite inconvenient. I wish they also offered a webpage. I hope you will consider that with Bigpay too.

DBS ain't too bad imho

Most countries have incredibly sub-par banking systems that work against the consumer. The UK is really on another level, though many countries are catching up.

I can't speak for Monzo but Revolut have presence in Singapore, and the product is incredibly feature rich.

On the note of PDF statements, I found that Okular on Linux was able to copy the table format that KBC Ireland use in their PDFs. That at least let me paste into a spreadsheet, massage the columns slightly, and then save as CSV for import into KMyMoney.

> DBS is hilarious in Singapore, the "World's best digital bank" 4 years in a row (forget the years) - who did they pay-off for that title.

Euromoney [1], which, on first impression, seems to be one of those content farms that make most of its actual revenue off selling tickets to award ceremonies.

[1] https://www.euromoney.com/

Having accounts in NZ, Aus, Singapore, and Taiwan.

Web banking in Singapore is bar none the best experience. It’s not even close.

Their deposits API ain't too bad. You can create bank accounts, get your transactions etc.

The six months limit is so hilarious for a so-called “digital” bank. They still use old mainframe at the back so they are obviously constrained a lot my batch processing still. It takes 2-3 days for most card payments to be cleared, which is weird given PayNow and other real time payments work in real time.

Thanks for the interest. It’s not really a super sexy project but it is useful for me. In addition, I really like writing software in Perl. I hope there will be a real resurgence of interest in classic Unix and Perl after people start copping it from the the towering complexity of other stacks …

Btw, the script currently cannot install Perl package dependencies. I will add code soon to make that happen (MakeMaker) so it becomes self-installing.

Feel free to add stuff for other banks!

I did a similar thing for Volksbank (a German bank) statements using awk:

https://github.com/airblade/volksbanker-awk/blob/main/vba

Perl one-liners are fine. But I’m not interested in writing Perl beyond that due to the towering syntax complexity.

This is exactly the kind of projects I like. The stuff where people make something to solve a personal problem.

There's a few things that can be tidied up though:

1. Your README instructions references `bin/import_csv` but the file name in the repo is `import_bank_csv`

2. You have a bunch of `use` statements in your `import_bank_csv` file which are not being used. Weirdly including `switch` yet you have a `elsif` block.

3. `import_bank_csv` also has a stray `1;` at the end of it. This is only really needed for Perl modules.

4. In addition to 3, some of your Perl modules do have `1;` and some don't. There isn't any consistency there.

I do share your love for Perl even though it definitely has a lot of hidden traps (I'd even go as far as to say it's not a well designed language by modern standards). But the fact I still hear about Perl scripts I'd written for jobs 10+ years previously, and running on production systems without any additional maintenance, says a lot about the stability of the Perl ecosystem.

Thank you for the feedback!

1. Spot on, I have fixed the README instructions :) It was written in a rush. Apologies.

2. You are right. I have removed the unused use statements. I had the switch statement imported as I wanted to try out the 'given ...' construct but it was easier just to use if-elsif.

3. Good point, removed.

4. Yes, absolutely. Thank you!

Having spent almost 15 years away from Perl in various other languages, I actually tthink Perl is quite elegant. You can do some pretty neat code in a very succinct way, as long as you keep your code structured and avoid too many clever things. The Perl stack definitely allows you to hammer a nail into your own foot, but it doesn't mean you have to. :)

BTW, I have now significantly improved the documentation and also added installation instructions. :)

Hi, I'm Singaporean and this is great! I use a similar tool for YNAB, though that requires a bit of manual work: https://github.com/aniav/ynab-csv

Unfortunately, there doesn't seem to be a way to export data from GrabWallet.

Brings back memories of why I liked Perl so much back in the 90's. Nice work solving this.

I'm curious about this bit:

  my $debitAmount = length(trim $fields->[4]) > 0 ? trim($fields->[4]) + 0.0  : 0.0;

That patterns repeats itself a few times

It feels like you could safely shorten that down to:

  $debitAmount=$fields->[4]+0;

Because the forced numeric conversion sort of does a trim on your behalf anyway, and "all spaces + zero" is zero.

That or a printf/sprintf: https://www.tutorialspoint.com/perl/perl_printf.htm

Thanks for the input! It is indeed a bit annoying I have to do that. Initially my code had what you suggested, but it caused a lot of "type casting" (in lack of a better Perl term) errors:

  Argument " " isn't numeric in addition (+) at lib/SGBankUtils/DBS.pm line 25, <$data> line 111.

Do you know of a way of supressing / avoid this error? I want to keep warnings and strict on.

Ah, it seems to be a warning, so it works, but is noisy. Something like what you're doing is needed then...I had thought space coercing to zero didn't do that.

Had the same tool for years but never made it public since there was. no expectation of any interest.

At some point I was thinking of dealing with HSBC PDF statements, but they supply them in non-split form so you need to OCR it first and then process. Was too much hassle at the moment

I've written a bare bones script to get historical transactions out of the pdfs statements. It was a PINA.

HSBC Singapore (and I believe Australia) have a facility to export transactions for an account to CSV. YMMV in other countries but it appears standardised. The output format is ugly-as, of course.

I’m in Singapore as well. Eventually settled on CSV, ugly as it is as you’ve mentioned

Reminds me of https://plaintextaccounting.org/

Shameless plug for a tool our team have recently launched to convert bank statements into spreadsheet format. Free for 30 pages: https://oscars-on-it.demarq.com/

I had the exact same problem with the exact same bank, HSBC. It led me into creating a web app that extracts transactions from PDF files.

https://bankstatementconverter.com/

I have worked on software which does the opposite. Convers sane json into horrible "narrative" format

Please do one for Binance.US

I am not a US citizen so unfortunately cannot help you there ... is the problem the same on their international one?

Reply on news.ycombinator.com