Comment by nzoschke | Hacker News Reader

Comment by nzoschke | Hacker News Reader

Thank you for this guide Kelsey.

I worked with countless apps and developers at Heroku on getting their apps running well on the platform. There was always one great mystery: why not build our apps a bit differently (dare I say better) to work in the cloud?

The database connection pattern is spot on. For any network resource, try to connect and if there is a problem retry with back off.

Also log the connection error events that a monitoring tool can notify off of.

I've seen apps that have the absolute worst behavior around this error that will happen. The worst is crashing the app in a way that triggers thrashing around restarts.

We had to build tons of complex restart back off logic into the Heroku platform to handle this.

I often wish app frameworks made this easier. I think most devs don't do these things because it is a chore for s problem that only happens occasionally.

But what if Rails baked this into ActiveRecord?

At one point Rails only logged to files on disk. We came together to add stdout logging to the framework.

You're welcome.

You have managed to capture the spirit of the post. The goal was to highlight areas where developers can take action, and how improving even the little things can go a long way to improving the entire system -- even the one you can't see.

In the early parts of my career I would often take pride in building complex systems to accommodate for misbehaving applications. Throw in some fancy Nagios alerts and a sleep depriving on-call rotation; I looked like a hero.

Then I learned how to write code.

This was the turning point in my career (2006). I was now brave enough to modify "legacy" production applications to take advantage of "new" infrastructure features like service discovery (use DNS records instead of IPs), and logging directly to syslog (asynchronously with proper ring buffers).

I was willing to learn any language too: Python, PHP, or Java, it did not matter because it allowed me to take action and contribute at the heart of the application.

I'm not saying platforms that also handle "misbehaving" applications or complex failure scenarios are unnecessary. I just consider those platforms as extra layers of protection, not a free pass to ignore building applications that take responsibility for reliability from startup to shutdown.

Reply on news.ycombinator.com