Hacker News Reader

Comment by timothygoltser | Hacker News Reader

While we try to form relationships with all of the brokerages we integrate with (including the ones we reverse engineer), we do still have to do the abuse/block dance with their protective systems occasionally. We've found that cutting out headless browsers has made it much easier to do this, for two main reasons:

1) Some brokerages have fairly sophisticated anti-screen-scraping protections, but their private APIs are comparatively undefended. 2) It's generally more difficult to create protective systems for private APIs, since there are fewer ways to fingerprint non-browser clients.

View on HN

I think "direct integrations" as a euphemism from "using reverse-engineered private APIs in violation of their terms" is dishonest.

That's a fair point - we've been trying to determine the best way to compress "using reverse-engineered private APIs in violation of their terms, but in practice brokerages don't really take enforcement actions against this," and "direct integrations" is what we came up with. We'll work on finding a better way to express this and are open to suggestions.

I think "unofficial API" is probably the most succinct phrase without being dishonest. A direct integration does imply some kind of professional relationship with the other company.

This sounds good - thank you for the feedback.

It's not as long as they expressly state there is no relationship or endorsement with the brokerages where there isn't one.

Perhaps. The current site and documentation appears to lack anything of the sort.

It’s a brand new company. Give them a chance to work these things out.

Well, they're taking shots at players like Plaid (who use majority direct oAuth integrations) while doing the exact same kind of skirting around companies that don't play nice and share permissioned user data.

Aggregators aren't scraping because they enjoy using headless browsers, the US doesn't let end users own their data and this is the industry workaround. I don't expect new players to change the system so much as I expect them to accurately represents it to those outside of fintech

> 2) It's generally more difficult to create protective systems for private APIs, since there are fewer ways to fingerprint non-browser clients.

There are? I think what you meant to say is “they haven’t worried much about key distribution, yet”?

In a sense. There are certainly protections they can implement against non-browser clients; what I meant is that it would be more difficult for them to implement the usual sorts of defenses you see against screen scraping since the surface area available for fingerprinting is much smaller.

We haven't seen brokerages expend significant effort on this, and the industry seems to be moving in the direction of providing more open access to APIs, so we're (cautiously) optimistic that we will be able to convince them to provide more sanctioned integration paths such that we don't need to continue playing cat-mouse with them.

What incentives do mainstream brokerages have to building public APIs? It seems to me data asymmetry is the name of the game in Wall Street and it’s no different for retail brokers.

“I ask my neighbours for some of their apples, but if they don’t give them, I go into their garden at nighttime with a big ladder…”

Wrong analogy. You’re describing theft. It’s More like, my neighbor planted an apple tree in their yard and told me, “this is your tree. You can pick any fruits off of it, anytime you’d like.” I told the jam making shop down the street that they can come get apples on this tree anytime if they give me some of the apple jam they end up making. My neighbor didn’t explicit permit me to do this, but also didn’t forbid me. I think? So now the jam making shop owner goes and grab my apple on my neighbor’s tree.

Maybe the solution is just to have the jam maker give the neighbor a can?

Except that it isn’t you inviting the jam maker, it’s the jam maker saying “I have a direct integration with the guy who manages your apple tree” , when they actually mean “I’ll dress up like you and pick the apples when nobody’s looking”

Reply on news.ycombinator.com