Hacker News Reader

Comment by misterbowfinger | Hacker News Reader

This is going to sound bizarre, but why do webhooks and not just an AMQP queue? I get that receiving HTTP POSTs is easier, but it just seems better to setup a publisher/subscriber relationship. That way, if a subscriber goes down, they can always catch up. And publishers can allow messages to sit in the queue with a TTL and max_size. It seems like a win-win for everyone.

View on HN

It's not AMQP (sadly) but something I've done previously is to have the actual webhook endpoint be as dumb as possible, doing nothing but accepting the payload (maybe with some very high level validation that the request was expected) and pushing it into a real queueing system.

This means you can handle all sorts of failure modes, not just the backend going down, but also bugs in the consumer that would otherwise result in losing the request. I've not tried it, but I imagine this is a pretty good usecase for AWS Lambda as it's a small bit of glue code.

I've used the same basic concept for accepting payment-received notifications (from a http redirect based payment gateway):

Read the transaction ID from the request body, and store it (with a date/time) in a table. A periodic process later checks them, and uses the payment service's API to validate the payment is valid and take appropriate action.

Shameless plug: https://requesthub.xyz is ideal for these cases.

That's pretty awesome, thanks for the tip.

One difference is in "dead connection detection". How do you know that your AMQP connection is down? At some level you're polling, whether that be TCP keepalive, application keepalive or something else.

If you're doing polling, you're actually back at the same pre-webhook place - polling their server on some timescale which is a compromise between latency and load.

Yes, a TCP keepalive is generally cheaper than an HTTP long poll request, but only by a constant factor.

It's a whole lot simpler to do secure cross-organization HTTP requests than it is to figure out how to have multiple AMQP subscribers from untrusted companies.

Though this is an approach worth considering for a bunch of local services, at least a subset of them, I don't see it working with 3rd party APIs. Consider something like Stripe — it is probably far more straightforward to invoke some HTTP endpoints, than to set up a huge infrastructure with millions persistent client connections.

Agreed. I believe that AMQP connections are best suited for wide and articulated local environments. Moreover REST HTTP endpoints are today's esperanto :)

Because anyone can throw another route onto port 443 if you already host a website. A non-HTTP protocol running on a dedicated port, despite often being a superior solution, requires extra effort to set up, if the hosting environment even provides that option.

Reply on news.ycombinator.com