Tips for Building Resilient Payment Systems (2022) | Hacker News Reader

Tips for Building Resilient Payment Systems (2022)

shopify.engineering

31 pointsredbell9 comments

I wholeheartedly recommend against using Shopify Payments, as they simply cut you off on a moments notice and they can’t explain why. This is the exact response we got from support:

> Regrettably we cannot share the specific reason or share further details about the rejection.

Like what? We applied, you approved, a lot of legitimate transactions happened and now you cut me off but can’t explain why? You really can’t trust them.

Who would you recommend?

For payment processing? Mollie has been good so far, however because it's an external payment processor we pay additional fee to Shopify.

I wish they talked more about maintaining consistency. How exactly does the reconciling of their records with their financial partner’s records work? Is it an asynchronous job that’s run on a schedule?

Also I’m sure they have dual-write consistency problems, how exactly do they solve that? 2PC? Event sourcing? Outbox?

Bank reconciliation is done using files. They will arrive randomly, typically on a multi-day lag. Then the payments are matched against the requests that were sent to the bank. The amounts _can_ and _will_ differ.

I understand it gets... interesting.

In general, reconciliation is done asynchronously. You send a payment out. The processor some number of days later sends out a file containing information about that payment. You validate that the payment the processor recognized is the same as the one you sent out. That's a high level summary.

Also regarding dual write consistency problems: they mention Google cloud. If they're using NoSQL such as Data store/firestore, they have distributed transactions. Another possibility is they must trust their payment clients to use idempotency keys correctly. If you get two requests with the same idempotency keys, then the requests are considered to be the same and therefore it wouldn't matter which one is processed first. If they're different requests with the same idempotency keys, they may have some validations.

I believe DynamoDB also has conditional writes that can essentially be "if entity X does not exist: write entity X". Then there is a race condition for different requests with identical idempotency keys, but if that does happen, it'd result in an error and someone would need to look into it and figure out why that was allowed to happen.

Thanks for sharing!

Lightning network did it better

This isn't constructive (or relevant). Are there any specific technologies or ideas that the lighting network could contribute to this article?

Reply on news.ycombinator.com