This doesn't sound good to me and translates to "our staff may not know what they're doing". Lack of or outdated documentation, lack of time to read it?
My job is currently mostly detective work, and it’s neither a function of incompetence nor laziness, it’s just a function of needing to prioritize business objectives over sacrifice-able things like documentation.
And that's perfectly fine. This post was published so GL folks can say "we figured out something tricky/interesting and our scale is what made it more tricky". I love figuring out "Frankenstein" type of bugs, errors and problems myself. What I dislike figuring out is something that shouldn't require any figuring out at all.
I read GitLab or Cloudflare's posts with pleasure. These doesn't seem to be written solely as technical recruitment "we do cool stuff" posts.
I left every single company which wanted me to spend a lot of time (weeks, months) to figure out their custom stuff due to no documentation and lack of people who worked on this and knew what makes the thing tick or what to do when it stops. Having noone or a single person kind-of knowing how to do something in a company is a managerial knee shot.
I value my learning/working time maybe more than an average person due to focusing issues. Spending it on figuring something which could have been documented but wasn't absolutely isn't my favorite thing to do. Weeks of detective work are weeks when I'm deprived of ability to learn something useful for my next job.
So while this GitLab post doesn't specify what kind of things are there to figure out, I consider stating that there's always something to figure out as unprofessional as it points out possibly unprepared staff.
Figuring out things you neither wrote nor fully studied beforehand, without breaking everything, is not some waste of valuable time, that IS the valuable expertise itself.
If you don't like doing that then that just means you're not a good fit for that role, not that there's some failing somewhere else that it's even needed.
It's not possible to document everything, and even if it were, no one would ever be qualified to do the job by this standard, because 3 things changed in production just while the on-call was making coffee.
This story was utterly normal and the only bad thing in it is just generally how tall and shaky and complex "normal" has become by now. But that's not anything any single company can do much about. Large web services need load balancing and orchestration and a lot of different moving parts, and every one of those parts are changing every minute because somewhere in the world a productive developer just committed a new line of code in something you use.
Well, one company can certainly make it worse, by e.g. ignoring years of API ergonomics around process spawning and give you a vaguely-specified "command" string, then pumping VC money into their marketing budget to make themselves an "indispensable" daily tool.
(Mostly I agree with you - just want to point out that in addition to the necessary complexity, the tooling around it also produces a lot of unnecessarily wrong-by-default behavior.)
It does.
What I meant by one company doing something about it was, in realistic terms, no one can do much about the fact that the rest of the world uses a lot of complex systems that aren't as robust as earlier simpler systems. You have to use the current tools and that's just how they are now. You can try to be less stupid as possible, but you can't change the ecosystem you need to live in and interoperate with.
Amen to that! It's basically how you start understanding how complex systems work in real-life and the lessons you learn while there can and will help you in every job you will have afterwards. Personally I also think it is what makes remarkable developers vs developers that just stay in their (usually tiny) domain.
My own sense here is that detective work is a deep part of being a software developer, because fundamentally we can’t escape the particulars of the problems we work on, but I guess that really depends on the complexity of the systems we’re working on.