Hacker News Reader

Comment by brightball | Hacker News Reader

This is an excellent point. I once had to deal with an external contract firm on a project that I was hired to fix. We had issues of production code breaking so badly that it brought down the entire server (N+1 query issue that triggered 50k queries).

The tests passed. When I emergency patched the issue and deployed it to production, the contract firm got mad at me for breaking their tests...to fix a production emergency.

It’s put me on guard again militant test ideologues with no concept of real priorities ever since.

View on HN

We generally wouldn't allow that either. We've ran into cases where emergency fixes cause even more damage (e.g., the system is up, but now it's processing payments wrong), so you have to prove beyond a shadow of a doubt that the test failures are irrelevant or less bad than the current incident.

Often times it's less effort/more expedient to make the change pass tests (or update the tests) than convince all of the stakeholders that what you're about to do is safe, but the break-glass is there if needed.

Maybe you'd call this a militant test ideology, but I think it's perfectly reasonable. Systems are complex, and people can get tunnel vision during a bad outage.

The way the tests were written in that case, it was hard coded to how the work was being done and not the result produced. Both the code and the tests were bad.

Normally, I’d agree with you though.

Be careful not to confuse militant ideology and local incentive structures.

Fair point. In this particular case the primary developer for them wanted to “publicly shame” me in Slack. Seemed much more ideology driven at the time.

That may have just been the rationalization for an ulterior motive.

Public shaming of a developer is rarely the right thing to do–and I would expect most design paradigms do not include a section on it ;)

> The tests passed. When I emergency patched the issue and deployed it to production, the contract firm got mad at me for breaking their tests...to fix a production emergency.

I get the point, but it depends on which tests failed.

Tests for unreleased features and trivial UX stuff is not the same as breaking a test making sure not every customer gets a 50% discount.

Reply on news.ycombinator.com