Where Banking APIs Fall Short
blog.svapnil.com
blog.svapnil.com
- files get persisted to disk by default. Usually we don't save every rest result to disk (though obviously, you could)
- files tend not to be processed at request time, but rather the directory acts as a queue. Usually with rest, we try to process things at request time until stuff gets too slow. Files always decouple
- multiple transports are the norm: SFTP, SMTP, etc. doesn't matter since it's not going to be processed at request time, but async. Think of "dropping it into a directory" as stripping off the transport layer the same way your web framework and reverse proxy abstract away http 2/3 for you.
- formats: much more variety than json everything. Back in the day, people liked to write custom parsers for everything. Nowadays, we like to let json be the bytes to in-memory data structure, and programmers recurse over that structure rather than spending their time ensuring their parser isn't ambiguous
It's true that REST APIs operate under the assumption of processing at request time, while not the case with files. That's what makes Bank REST APIs so tricky - processing doesn't ever actually occur at request time.
There are so many crazy formats that are supported. Simpler is better imo
HTTP has provisions to enable async semantics between the client and the server. I would always advocate for teams to return 202 Accepted responses with a Location header containing the absolute or relative path to an endpoint that can be interrogated for status of the underlying async operation. This can work fine for lower-scale services (or even higher scale services with rate limits enforced at gateway). You POST'd a payment. You get 202 Accepted with some Location value like (simplified) `/payments/<identifier>` for which GET can be used to retrieve current status (e.g. pending, failed, committed, etc).
If a polling model is not desirable you can also use webhooks and inform the caller whenever a given task is finished.
Both of these strategies work fairly well in practice in my experience. However, I Must agree that simply dropping a big file off at an sFTP share and moving on with life is certainly "easier" from the client perspective. But...do you then roll your own mechanism for checking status? Why not just use HTTP semantics that exist already?
I agree that simplicity and open standards will win. SFTP is straightforward and has worked well in the past. In banking and finance, SFTP is what powers most programmatic corporate banking. Eventually, I think we'll see SFTP and message queues (like SQS) used more often as processing speeds up. I believe message queues are what powers much of real time payment processing already (https://news.ycombinator.com/item?id=36805571)
Files definitely have limitations (they’re almost always a single table of data—it’s hard to do data structures), and that seems to be driven by the need for someone in the chain the be able to open it in Excel.
This is key. I know from experience that the ability to fall back to emailing CSVs around and having operations folks manually process them in Excel is required way more often than your average tech oriented person would ever expect. Plus a lot of processes that might appear to be automated at banks turn out to have someone doing a ton of manual work as part of their daily process. Things gradually and haphazardly eventually get automated but don’t necessarily start that way. Banking seems to oscillate between strict standards with robust technology and random tasks developed when a specific need arose only to never change for long periods of time.
Would be curious what the author thinks of initiatives like Open Banking, which don't seem to be a big topic in the US, but are in the EU and Canada at the moment.
My expectation is that banks should be abstracting away the 'under the hood' part, and so the future would look like an API.