Heroku REALLY screwed up today
Today, predictably enough, they broker the ability to restart jobs, then they started having elevated dyno errors.
But today they really screwed up by ALSO making it impossible to scale down worker jobs to ensure no background process (like emailing) get fired off until the dust settles.
We shut down all our worker processes, the heroku console confirmed it, but then we watched the (supposedly off) delayed_job workers blithely keep firing off jobs...
Downtime is one thing, it just comes with the Heroku gig.
But Heroku need to cease the irresponsible practice of scheduling routine maintenance during peak hours now that they've proven they are unable to ensure apps can do an orderly wind-down if/when teh inevitable breakage results.