Hacker News Reader

Comment by tasn | Hacker News Reader

I don't think they do, our whole job as an API company is to have uptime (and redundancy for uptime). I think it's the same with every other API company, such as Sendgrid, Twilio and etc.

User endpoints on the other hand, fail all the time, and often require a few retries.

With that being said, we plan on having client side (in our libraries) redundancy (try another endpoint if one fails), and ways for you to gracefully handle errors locally for users who need this.

Edit: I realise this comment was a bit sloppy, added some context in another comment https://news.ycombinator.com/item?id=26400970

View on HN

It's not just your infra though. Say I'm a client sending you webhooks. My internet connection (if not in cloud infra) could be down, or my cloud provider might not have a route to your delivery endpoint due to a routing or similar outage. If those requests block, the client's app is going to hang or back up. If they don't block, they have to go somewhere (perhaps journaled with a task that can be executed on demand or on a recurring schedule).

My suggestion would be not only a library, but a local queue that can be managed with the library. Also, having some experience with working at a no code provider previously, ensure that it's trivial to query your service for what webhooks you can confirm were received by your infra and then delivered, perhaps by UUID.

Last, make sure it's dreadfully difficult to make changes to your systems where data loss could occur (messages dropped while returning 200 to counterparty systems, for example).

It's a good point, and actually some of our users already have this local queue implementation anyway because they want to send webhooks asynchronously. So it's definitely something to keep in mind. We already make it easy to query the service for what webhooks were delivered, but we also let you use the API with idempotency keys to ensure messages are not accepted twice.

I agree about all the issues you outlined though. One thing we plan on doing in terms of internet being down and cloud provider not having routes: have a lot of endpoints spread geographically in every region, and maybe in the same/nearby data centers.

Thanks again for the feedback!

For what it's worth, Stripe's Idempotent Requests are a nice way to handle this: https://stripe.com/docs/api/idempotent_requests

We just send the same requests over and over until they're successful. It's relatively simple to use.

Yeah, we also support idempotency (the event ID is used for that). I'll add it to that comment, I forgot to mention it. :P

Edit: ahh, I already mentioned it, it's buried in my comment.

> I don't think they do, our whole job as an API company is to have uptime (and redundancy for uptime). I think it's the same with every other API company, such as Sendgrid, Twilio and etc.

That doesn't sound realistic to me. Sendgrid, Twilio, and all other APIs have downtimes and other problems, just check their status page: https://status.sendgrid.com/history.

I always had to implement some retry logic in API clients at some point or another. That doesn't mean your service doesn't bring value but it's not enough to just expect it to work because it is focused on uptime. When implementing a service that requires external dependencies I would always expect them to fail and try to design with that fact in mind.

I agree, and I think my original comment was a bit sloppy.

We already have users that have their own queue system for sending us messages because that's how they deal with similar APIs too.

So I definitely see the problem, what I meant in my comment is that it's similar to other API companies, and it's not unique to us!

It's not always your service crashing you need to worry about; reliability also includes the network (and all of the routers) between you and your customers.

Yeah, I replied to a sibling comment. You are right, I was a bit sloppy with my message. I was just trying to make the point that it was similar to other APIs, and that we take uptime seriously.

Reply on news.ycombinator.com