Hacker News Reader

Comment by whalesalad | Hacker News Reader

PagerDuty feels like abandonware. It hasn't changed in the last 4-5 years.

How much can it really change though? It’s a fairly simple concept. I always considered the cost is paying for trust and reliability. I know a text message might fail. I want those alert redundancies and I don’t want to have to create that system.

I tried some of the competitors out to reduce cost, but UX was definitely lacking (opsgenie for example). We stuck with PagerDuty.

That being said, as a potential investor, I don’t think it would be hard for these companies to reproduce what PagerDuty does, or anyone. They just happened to be one of the first but companies with large pockets, especially Atlassian/VictorOps can make a lot of headway here.

I was surprised to see they aren’t profitable, but I guess a lot of the spend is marketing/sales and the long cycle to get those enterprise customers. Especially the $100k ARR ones.

The last time I looked at PagerDuty (a while ago) it didn't have any kind of "calling tree" equivalent for validated notification escalations.

Couldn't really see how that could be true, so I'm assuming I didn't understand the product.

We use multiple schedules with Pagerduty, and when an escalation policy is triggered, it can route to multiple additional schedules. So let's say you can escalate to an SME schedule, if a certain incident is not resolved within a certain amount of time.

They have a configuration tab for "business services", where you can set up dependencies between services, but we're not using that, so idk how smart is that.

When I call my insurance or bank, I get a pleasant voice interface.

When PagerDuty calls my carphone (or worse, my motorcycle helmet communicator), I get an inhuman civil defense robot from 1980 that only accepts touch-tone inputs.

But, it absolutely never fails.

I don't know if I would go that far, but the service is quite reliable. I have experienced some spot issues with message and alert delivery over the years, but overall, reliability is strong with this tool

This is not true in my experience. I led the monitoring team at my past job and we spent a non significant effort monitoring PagerDuty due to often delays in ingestion and notification. If you page through their status page[1] they have a problem with delayed ingestion or notification almost monthly. If people actually monitored they'd be surprised how often they miss SLA.

[1] https://status.pagerduty.com/history?page=1

I wish I could opt in to that for all automated communications. Fighting my way through voice-recognition menus is always a pain in the ass that makes me wish for the telephone menu systems of the 90s.

I like having both options!

Better than old school pagers from the 80's which just had an earth shattering sound.

I remember being on call for Telecom Gold (uk version of Tymnet) got home from London late pager fell down the side of my bed and I didn't notice.

Later that night at 3am it went off like a small nuke and it literally took me 20 mins and a coffee to gather myself to be able to call the Operators and then logon (dial up to a x.25 PAD)

When Pagerduty calls me, it's because I've missed the push, email, and SMS notifications. The phone ringing gets my attention but I usually hang it up and open the app.

If I'm driving, secondary oncall can get it.

They're spending $30M+ on R&D, so if you feel that way as a customer, that's not good.

On principle alone I wouldn't give them my money ever again. I'd engineer my own lightweight solution before then or use an underdog competitor. I have a sour taste in my mouth for their meteoric rise to success/IPO versus their pathetic investments (or lack therof) in product improvement. It really sucks to use. Reminds me of Engineyard (also abandonware).

We tried using one of their competitors and strongly disliked it, enough to go back to PagerDuty in under a year.

I'm sure there are other options we could have tried, but it seems to do what we need pretty well.

I feel that while OP is correct - PagerDuty hasn't changed much - engineering my own on-call system is the last priority of pretty much any business. It's one of the things you don't want to do yourself.

Additionally, PagerDuty works extremely well and I can't think of any additional features we need right now.

Honestly, if your response to critical ops notifications is to engineer a lightweight solution as a part time pet project, I don't want to work for or use your product.

If you are talking instead about hiring an ops engineer to build your custom lightweight solution, I would ask why you're spending 100k+ all-in cost on an engineer to do something you could pay PD to do for cheaper.

Well compared to victorops it's 5 years ahead. We recently switched off victorops to PD. The victor-ops scheduling UI is crazy clunky. Also PD seems to page faster, and the mobile app is better.

I went from a company that used PagerDuty to one that used VictorOps. Exactly as you say: the PagerDuty UI is much better than the VictorOps one.

For me it's mostly about changes to their phone/watch apps and schedules management

And check this https://www.pagerduty.com/whats-new/

Agreed, too many good open source solutions now. No one thinks of alerting as paging anymore. You can use email to text now, chat solutions like Slack/Mattermost/etc. You no longer need an SMS gateway and if you do then Prometheus supports Twilio. All PagerDuty has going for it is a sales team trying to get managers to sign up for a service before exploring alternatives.

Out of genuine curiosity, do you have a list of good alternatives or some reading material on "paging" best practices? My team uses PagerDuty but I've never really questioned if there's a better method out there because the system PagerDuty replaced for us was so terrible that PagerDuty seemed like a godsend.

Google's free SRE handbook has some info.

You guys are out here giving very strong dismissals of a very good service, and then referring us to a resource you wont even bother to link? Maybe you can link the actual resources to prove your argument instead of assuming you are correct and dismissing people to figure it out on their own. Usually, the standard for conversations here is held to a higher, non-twitter level, and I for one enjoy conversations held to higher standards.

I think they must be referring to this: https://landing.google.com/sre/sre-book/toc/index.html

I don't see alternative services on this page (https://landing.google.com/sre/sre-book/chapters/monitoring-...) Do you have any alternates you would recommend?

The value of Pagerduty is not in its notification, but the workflows around it. Alert in the middle of the night? Don't page unless its critical. Management of on-call schedules, escalate when on-call engineer doesn't answer. We use OpsGenie for this (in addition of sending messages to a chat, but off work that is usually muted or not installed on engineers phone), but PagerDuty has the same functionality.

I swear half the time my Slack notifications don’t go off on my phone. Even with background refresh on. I would never trust that service for critical alerts. As in, you need to wake up and fix this now.

I’ve run my own monitoring solutions. My own Asterisk systems. Even my own Twilio integrations. You are grossly oversimplifying what is needed to create and maintain those systems. For many, PagerDuty is absolutely worth the cost, which is why it’s so popular. People will pay for reliability they can depend on so they can focus on their core business.

I think by default slack has Do Not Disturb enabled from 10PM to 8AM. May want to double check that!

Figured that was it, but it’s happening during my daytime outside of the DND.

Slack notification delivery is definitely not what it used to be.

You mean to say it's slacking off?

https://support.pagerduty.com/docs/platform-release-notes https://support.pagerduty.com/docs/release-notes

If you want a really crappy experience, try something like Everbridge. It’s a platform for calling parents to tell them today is a snow day masquerading as an IT alerting platform. You’ll be begging for PagerDuty in weeks.

What are they doing with all those R&D dollars?

Timbits and Crown Royal. It’s not going to the product that’s for sure.

Man, if I was them I'd be going for Bearface Whiskey and Pie Commission beef pies. But that's just this humble Torontonian's opinion.

I've been a Pager Duty user since 2011. It's a good product and fulfills its need. I don't see it as a product requiring 8 figures of R&D, so seriously what are they doing with it all?

FWIW, you are close, but substitute some sort of bourbon or scotch instead of Canadian whiskey :)

EDIT: this was a joke. I worked there so I know where the R&D dollars went: to quality engineering. But it is the damned truth that we drank Bourbon and Scotch, not Canadian whiskey

the fireball was canadian (though maybe not "whiskey") ;)

Reply on news.ycombinator.com