Should You Build a Webhooks API?

If you're using a web hook to notify your users of changed content, the WebSub standard (https://www.w3.org/TR/websub/; formerly PubSubHubbub; currently a candidate recommendation) is a good option. It solves a couple (though not all) of the problems of web hooks listed in the article, including:

- Uncertain security: signing notifications is in the spec

- Development and testing: ideally, with libraries that implement the spec, you can assume that the websub part will 'just work' and only need to test your handler function.

- Misbehavior is onerous: subscribers need to actively confirm their subscription every once in a while. This might seem complex, but it allows for periodic checks if the server/client is still functioning as expected, and ideally can be handled automatically by libraries.

- Retries: the spec leaves some room here, but again, a library can handle this.

As I decided to use WebSub as an alternative for a 'plain' webhook for a project recently, I wrote a library for the Flask web framework: https://github.com/marten-de-vries/Flask-Websub. I personally hope libraries for other frameworks will follow now the spec is moving less.

We built Auth0 Extend [1] to address with many of these concerns. Our hypothesis is that the logic that implements these extensibility points should live in the host platform.

For several years now, we've had this sort of hook-via-code in the Rules engine on our Auth0 identity product. It's been such a powerful enabler for many different integration scenarios that we thought that other companies may find the technology useful for their platforms.

Since committing to building Auth0 Extend, we've seen Twilio launch their functions [2] and just today CloudFlare launched its Workers [3]. It seems like the this is an approach that is in its infancy but that is here to stay!

1: https://auth0.com/extend/developers

2: https://www.twilio.com/docs/api/runtime/functions

3: https://blog.cloudflare.com/introducing-cloudflare-workers/

I've setup a few webhooks over the past few years, and one of the complications that people rarely talk about is the data that is passed to various webhooks.

As the article alludes to APIs change over time, and so you can find the data that is submitted to you changes. In my case I setup an end-point that responded to push events from various code-hosting services.

I found that the three "main" git-hosts sent widely different data in their webhooks, and updates we often painful. (Github, bitbucket, and .. something else. I forget now.) Trying to have one end-point to handle all services was a recipe for frustration and failures. Even parsing out the basic details was hard:

* This is the SSH repository URL of the thing that was committed to

In the end I moved to self-hosting git-repositories, rather than allowing users to point at my webhook. That allowed hooks in the master-repos to fire.

(In my case I trigger DNS updates when git commits are made.)

Good tips! I actually just worked on detecting dead webhook endpoints, as I've had a number of users use ngrok for local testing, create hundreds of events, and then disable ngrok--and much like Stripe, I retry with an exponential backoff over the course of a few days. Needless to say, my logs were filled with failed webhook request errors and I had to figure out how to limit that from happening in the future. Very interesting list of alternatives to webhooks as well.

I guess the section on Retries is closest, but we're looking at moving from a system with straight in-process hooks to one using webhooks, and a complication is we now have to provision a webhook endpoint with very high reliability to accept the callback and (effectively) call the old hook.

The thing is, all the hook does is put a message on a queue…

For those interested in technically easy webhook consuming endpoint server check out https://github.com/adnanh/webhook

I would love to see some of these APIs replaced by WebSockets. It would allow building your own client for some services much more easily while maintaining push notifications

Webhooks should not be mentioned without also mentioning SSRF vulns. Because most companies use their firewall as a defense permitter, these attacks can be devastating.

Side note: their music is so good. Paired with their actual tech and vision, I think the music actually makes me want to work at SpaceX more than if paired with something less epic.