Comment by Lazare | Hacker News Reader

Comment by Lazare | Hacker News Reader

Parent Full thread

That jumped out at me too.

Obviously they were under a lot of pressure and it's easy to stand here and criticise, but...

...if my site is slowing down with load or usage, I'm not sure how you make the jump to "I should update my UI libraries!". Angular 4 isn't getting any slower, so best case is you've got some unknown performance bottleneck in your UI that is somehow causing 30s page load times, and which just happens to be fixed in Angular 6, and you don't accidentally add any new issues when you upgrade.

Conversely, it feels like if you're struggling with "slow load times" on a SPA, the first thing you'd do is open the network tab and see what requests are being made, to what, how often, and how long they're taking.

Grasping at straws does seem to be the right metaphor. (Or maybe the old chestnut about the drunk dropping his car keys in a dark parking lot, then looking for them under a streetlight, since it's too dark to find them in the parking lot?)

I'm happy for the team and it sounds like things are going great for them, but wow, that was an almost fatal bit of blindness. On the plus side, I bet everyone involved will check for inefficient database calls first next time. :)

That’s why you need good humans who’re expert at responding extremely well when things break. It’s one thing to prep for a whiteboard interview, it’s another to intuit that needle in the haystack.

It's only about finding needles in haystacks if you don't have sufficient instrumentation and monitoring in place. For most production issues your tools should be able to guide you at least the first 75% of the way to your issue, even if they can't usually offer a good fix (though sometimes they can, such as when new relic points out missing db indices causing unnecessary full table scans).

I agree. It's also about trusting developers' intuition when debugging problems. We recently (past few working days) went through something similar; where we had a problem blocking us from a release, and people scrambling to figure out the problem.

We have some software that was returning different results from different environments, and we couldn't figure out the problem. There was a lot of panic in the room, from upgrading and downgrading Maven dependencies, building things inside and outside of Jenkins, and all sorts of random things.

We kept telling the project leadership that we're poking at the wrong part (intuitively), but they kept pushing. I've had to explain how Maven works, how building on Jenkins doesn't differ to building from our IDE's, etc.

It's only when we asked for isolation from the (human) elements, that we had the freedom to properly debug.

In the end, an unstable sort was the cause of the issue. We were taking the last element from an array, but not sorting the array first.

All of the stuff we did since last Thursday to Tuesday evening didn't help us.

So, I agree, you need good humans who are good at responding well when things break.

I've been in situations where the whole developer team is trying to solve a production bug, and the managers have never tried to tell us where to focus.

That wouldn't make any sense at all.

Every developer would have their own guesses that they would need to explore and validate, sometimes there's some grouping around where the focus is, but there's usually one guy that's exploring a totally different area to find that bug.

On the other hand I've spent hours debugging an issue in a C++ program I work on only to eventually find that it was a bug in the oldish version of GCC we were using and simply upgrading the compiler would have fixed it.

It's entirely possible that they could have spent hours or days debugging their issue only to find it had already been fixed.

But after those hours you knew exactly what the problem was and exactly what the solution was. You were in a much better position and could make better judgement calls at that point. It's better to dive in and figure out a problem than to just randomly upgrade things.

This isn’t really the other hand. This is saying exactly what OP is. Debug your system and fix the bug when you find it. Don’t randomly change things hoping the problem goes away.

Reply on news.ycombinator.com