Show HN: NLP and NER-powered beautiful bank transaction feed
try.herondata.io
try.herondata.io
Our new approach consists of using custom Natural Language Processing and Named Entity Recognition models to predict a) what substring of a bank transaction string represents a merchant e.g. "FACEBK" and b) which canonical merchant e.g. "Facebook" this substring corresponds to in our merchant DB.
The toughest steps were to manually label thousands of bank transactions very carefully to use as training data, and to deploy the models in a production env where we need API response times to be under 200ms (usually what's required in order to incorporate this API in a payment auth flow).
We always optimize for accuracy (>99% currently), because we never want to return an incorrect merchant, but with this new approach our coverage is now at 80% of all bank transactions we see through our system.
We would love any feedback and comments, and also happy to answer any questions about the product or how we productionized it!
We have trained our custom NER to detect merchants using millions of transactions as training data rather than a general NER.
We have tested with a few general NER models but they weren't detecting merchants properly. We haven't tested with LSTM/GRU yet but that's a good suggestion!
In other words, in the example on the homepage, is all of the returned data except the dollar amount static data from a DB once you have matched the transaction to the merchant (Google in this case)?
And then sort of unrelated, what are your long-term plans? If step 1 is perfectly hydrating a merchant to a transaction, what is step 2 (if there is one)?
Yes, we match the raw string to a merchant with extra static fields (URL, icon, logo, Merchant Category Code).
In our mind, merchant enrichment unlocks many possibilities. The most obvious one is to have a nice and modern bank transaction feed. But our customers also use us to lock company cards to particular merchants, group by merchants to see where people are spending the most, and more. Would love to hear if you had other "step 2"s in mind!
The average response time for this endpoint is ~200ms, and we guarantee this latency for up to 100 requests per minute.
In production we currently handle ~1 million transactions a day with no problem.
Does this answer your question?
This currently supports merchants present in the US, UK and Canada mostly!