What I’ve Learned in 45 Years in the Software Industry (2021) | Hacker News Reader

What I’ve Learned in 45 Years in the Software Industry (2021)

bti360.com

273 pointscolinprince89 comments

Good list. With a paltry experience of 14 years, I would add one silly thing that kind of feels too obvious to even add:

When there's an error or exception, read the error and the stack trace thoroughly and figure out what it means even if it means googling. Lately I've noticed that younger generation doesn't really understand error messages even in the simplest cases and people end up in wild goose chases when the real reason was incorrect permissions or something really mundane.

The error messages, at least in front-end, have gotten worse.

In 2012 I had a stack trace that I could read and eventually figure out what happened.

Nowadays it's async framework method calls upon async framework method calls and that is an improvement in the tooling, because it can now glue them together.

Next-gen frameworks are undoing that mess, but god help you if you use some third party library for e.g. state management.

I like to rag on c++ for having terrible errors, but they're way better than they used to be.

> Lately I've noticed that younger generation doesn't really understand error messages even in the simplest cases

It was ever thus. I suspect what’s happened is that lately you’ve become the older generation.

I doubt that. I grew up learning to use a C debugger and interpret core dumps right from my undergraduate school. Knowing the error and inspecting the stack were something I just had to do. In the absence of google and stackoverflow the ability to inspect your core dumps and debug the stack were essential.

Bro I used to provide FreeBSD support on IRC over 20 years ago. People don’t read error messages.

Sometimes the error message is spurious and the solution (such as installing the right version of a software package) is well-known. Maybe someone knows this and can solve your problem instantly.

Other times, perhaps frequently, the error message is actually telling you something important.

Many people get stuck in the second case thinking it's the first case.

Isn't that covered by this?

> When there's an error or exception, read the error and the stack trace thoroughly and figure out what it means even if it means googling.

The infuriating part is when there is no attempt to parse the error and just fire off a screenshot in chat and expect someone to do the first part too...

I disagree. Stack traces usually only have 2-3 lines of signal. I never read the stack trace thoroughly because it's mostly noise.

After learning this lesson the hard way, my co-founder and I would refer to this lesson as “RTFE” - Read The F*** Error”

And if the error message is really incomprehensible, you missed closing something on the previous line.

In their defense the death of manuals and the death of the web/Google makes many error messages a rabbit hole of bullshit.

Two bits of advice I'd add, from 30 years in the industry:

* A colleague once said to me, a few years back, "become an expert in one thing", even if it's a small thing.

* "My superpower is reading the manual"

I think "reading the manual" can be be interpreted not just in the literal sense of whether there's an actual manual, but also in the metaphorical sense, to make sure we're considering easily available sources of information. Great advice.

Yep this is a really important skillset.

I've written a post around this topic because you can do this in a systematic way to learn pretty much anything https://nickjanetakis.com/blog/how-i-quickly-find-answers-to....

The code is the best manual you'll every get in my experience. There's no substitute when working on a codebase.

When I'm introduced to a codebase I'll be doing significant work on, I'll often spend hours at a time just reading the code. Start at main() and skim until you get a feel for the overall architecture, then go figure out whatever subsystems seem mysterious to you. Rinse and repeat. Makes a huge difference when making changes, because that requires reading code anyway, and you'll have a much better idea of what code to read/change.

And it will for all future make it impossible for you to distinguish official API and implementation details.

That just comes with the territory of being someone who has to understand the implementation details. No getting away from that.

Code also has the advantage of not getting out of date with what the application actually does. Invariably, documentation outside the code base often rots over time until it reaches a point it's outright misleading.

Yup, that's a good reason to keep documentation in the same repo, and to update documentation in the same commit as the code that causes the change.

One scenario where I find external wikis better is when a change encompasses multiple components and you want to present an overarching view to the user + want to refer across sections. This can be managed via something like github pages for each component but the cross referring across multiple repos adds more friction. Plus to present the overall view, something like a synthetic "doc-repo" will be needed that has landing pages that connect the component-level pages. Or we can have a wiki that does the integration by refering to component level github pages but at that point why not just do the whole thing in wiki much more conveniently.

My experience is that it basically comes down to developer discipline. A disciplined developer will update the docs regardless of whether the docs are part of repo or external and vice versa.

Imo reading gets you nowhere and is inefficient. Key piece is to add some form of self explanation or diagram to map the overall picture, since it takes a lot of repetition to understand the code flow but only hours to come up with a data flow diagram and revisit/update when needed. Even faster when someone on the team has already done it.

Maybe its just me but plain text does nothing for me so i convert it into visual elements.

> Start at main() and skim until you get a feel for the overall architecture

Man I wish that worked for Java ;-)

> even if it's a small thing

How small is small? Some small fields are quite big when you factor in all global experts.

Something that you, personally, could be reasonably expected to know pretty much everything about, whether that's an internal system, a public API, or tool. It's up to you whether the topic is transferrable.

This is pretty like the quote from Thomas Huxley: "Try to learn something about everything and everything about something".

But I think people should choose topic carefully it should be fundamental, maybe it's soon unfortunately gone if you choose a trendy and short-lived tech.

Never heard that quote before. +1

"How small a thought it takes to fill a whole life!" – Ludwig Wittgenstein

My only superpower is knowing what fad is going to make a stack of cash and riding it. The only thing I've actually honestly believed in and honed recently was Go programming.

Mind sharing what fads have been the most profitable for you?

I can’t take any credit for having some omniscient insight but I was lucky enough to start Java development in 1996. It pretty much gave me a career.

Same deal, 2001 when C# was in beta.

That's worked out.

I've done a wide variety of things. I try to keep two disparate things on the go. Visual Basic between 1992-2000. C# between 2002 and 2015. SunOS/Solaris admin 1995-2002. SQL Server 1999-2010. Linux admin 1998-present. Python 2005-present. Cloud infra/AWS 2012-present. Go 2015-present. I also did some random PHP and Perl things for stupid money in the 00s. Now I'm Linux, Go, Python, cloud infra.

What I really want to do is be locked in a room with a niche project I can look after until I'm dead a this point. But that ain't gonna pay the mortgage off...

I too have recently started diving into Go in a more comprehensive way. Any ideas about why it seems to you right now is a good time to cash in on golang?

My best guess is the ascendence of Kubernetes has made reading Go rather instrumental; and as a by-product, writing Go seems much more approachable.

Not GP, but: Critical mass. Accepted in many domains as a valid first class solution. Memory safe. Has that depth of modules now that you can `go get` most things off the shelf. Easy to get a team up to speed, leading to wide adoption.

It is like the Python of statically compiled languages to me, but with way more safety rails and things you need as your code base grows.

There is just demand for it in general from what I see.

I'm not really cashing in on it. It has become my sidearm. I think that's the best way to describe it. When I want to solve a problem it's just there, trustworthy, reliable and efficient.

What it does is allow me to circumvent demotivating and productivity draining pain which is seen elsewhere which is pretty much summed up as: shitty build tools, runtime environments, libraries, containerisation, IDEs, concurrency approaches, test frameworks.

Really it's a tool that allows me to actually get stuff done.

It's pretty much what I've wanted for years: a memory safe, GC'ed C with a stdlib that wasn't conceived in the 70s.

The stdlib is really great. The things you can cross compile not worrying about glibc all day long is amazing.

Manual? One should be so lucky.

Manuals used to be great when software was stable... that is new versions we're not updating online at any given moment. After a few updates these days the manual might as well be for different software totally.

Exactly. I yearn for documentation that doesn’t require me to essentially run experiments to figure out what a library does. It is mind blowing how much time I’ve wasted in my career figuring things out that could have just been explained by accurate documentation.

Often the "manual" is just a computer-generated hyperlinked soup of tautological sentences like: this function does what its name suggests it does.

Is not that anti "antifragile" by design ? Like a being which is very well adapated to its environment: it can not survive if its environment changes.

Forget manuals, simply being able to read a stack trace puts in the top 10%.

Yeah, and being able to use Google, too...

And actually being able to post the five or six lines of the stack trace that are relevant, without the fifty or sixty that really aren't.

Those are great points. Generalizing a bit:

* Identify your superpower

What special mojo do you bring to a problem that no one else on your team can? Recognizing and leveraging your superpower can make running w/ a team of true overachievers a lot more pleasant.

RTFM (classic advice to a Unix newbie)

+1 for SC's 7 habits

Early in my career, my manager gave me the feedback that sometimes my eagerness to participate outsized my ability to understand. I never meant to suck the air out of the room, I was just so excited to build cool stuff! He recommended reading up on Stephen Covey and suggested that when collaborating on designs/debugging/etc. as a team, to practice being the last to speak. Life-changing advice.

I like to summarize everything said and add my own conclusions. It only belongs at the end.

Sounds like a good manager, giving useful feedback!

>He recommended reading up on Stephen Covey and suggested that when collaborating on designs/debugging/etc. as a team, to practice being the last to speak.

I'm unfortunately in the extreme end of that advice. I listen to my teammates but I rarely contribute anything worthwhile, not because I don't want to but because I've nothing to contribute. In two recent sessions about database design for one project and integrating SSO into another, I literally had nothing to say.

I think its a balance. While your questioning and speaking up should minimze the disruption, a junior person needs to learn to speak up even if the idea is totally dumb and crap. You cannot just listen to others speaking and learn to think and speak on the fly, just like you cannot look at swimming videos and learn to swim.

You don't need to be the last to speak. Let the usual "speakers" do their bit first so that you know the main points that remain to be covered. If that means being last because everyone in your team is a "speaker" then you need to decide if you are ok with that dynamic. If no one on the team is a "speaker", maybe you have an opportunity.

Previously:

What I’ve Learned in 45 Years in the Software Industry – https://news.ycombinator.com/item?id=25658216 - Jan 2021 (345 comments)

> Seek First to Understand

Yes, this. If there's one issue I've seen again and again limiting engineers it's them fixing some part of a system without understanding the system as a whole, while assuming that someone (the code reviewer say) does understand it. It's a machine for manufacturing bugs and technical debt, and it keeps the engineer in a junior mindset.

I always liked this one:

"Bad programmers worry about the code. Good programmers worry about data structures and their relationships."

-- Linus Torvalds

Honestly I think #3 should be #1. And with that, entire frameworks and languages should just be thrown out the window by the entire industry.

I'm sick of working with juniors who don't know how to find the simplest approach to problems without pulling in yet another 3rd party dependency ... or more likely want to write obtuse code because it makes them feel flashy. Just makes me want to shoot myself in the head.

one bit of advice about contracting in the uk - given to me ages ago when I first started contracting is:

"It's like playing a game where the rules change without you knowing and people cheat"

Sounds grim. Any examples?

Not the person you're responding to but I've done a fair bit of contracting, although in the US.

A perfect example is scope creep, which happens with all sorts of clients. And, as much as we all hate it as devs it's just the way the world works - stuff changes. Some clients will work with you to update the spec and make sure you're compensated for extra time, where others will just expect you to "roll with the punches" effectively working for free.

A personal anecdote of this was when a medium-size company switched payment processors half way through our contract. I was building a custom subscription experience over a 3-month timeline. I had already finished the specified payment parts when they came to me wanting to switch it out for another provider... When explaining that change would effect timeline, and that I would need to be paid for the extra labor they got very combative. I pushed back on them and they threatened to pull the entire contract, sue for the 50% up-front (if not reimbursed), and even heavily alluded to slandering me among other customers of mine in their niche industry. They strong-armed me into doing the work because it was either losing two weeks redoing payment processor integrations, or losing over two months of work and potentially taking a huge reputation hit.

Over the years they would reach out for new work, but I never contracted for that client ever again. Gotta pick 'em carefully and I don't tolerate abuse.

On a contrasting positive note... Other clients understand scope creep involves extra work, and even proactively offer to pay for the labor/update timelines. For these folks I will bend over backwards because of the respect.

---

So back to the "rules change" and "people cheat" - 100% accurate. Not all clients are like this - but, you have to learn to gracefully handle and eventually cull the ones who are.

> Solutions should be as simple as possible. Assume the next person to maintain your code won’t be as smart as you.

This resonates. But I find it hard to tell to a bright person that their code or solutions are too complex.

I've found it harder to convince managers that code should be simpler. Sure a solution works, but it's a mess. This is especially true of my own code.

Open with a solution but be very open about its flaws, assumptions, and invite change. This is a great way to stimulate teams to start thinking.

I’d like to add: accept imperfect solutions but try to think few steps ahead.

It’s an art to produce something that lasts long but doesn’t take years to build. Tech evolves rapidly.

I'd like to add: There's way too much stuff taken as dogmatic best practices when they're really not. Things change, improve, and there may be better approaches already in existence but are ignored because of some high profile person in the community. Keeping an open mind is key to being able to improve and use better approaches.

For me it's : 50% of software development occurs outside the computer

90% would be more accurate.

True: marketing, sales, support, training, finance are part of holistic product the customer is buying into. The customer themselves is part of it too!

I love this content! I love his wisdom! But my goodness, it's like some of these websites go out of their way to not be readable at all.

Not his fault your terminal supports more than 80 character rows.

(If you got Firefox click the reader mode in the right of the URL bar).

(Tried Reader on Safari iPad; Wow!)

Honest question: what was the problem?

The photographs? Photographs are almost always junk in an article, and this article was no exception, except it had the saving grace that this was his swan’s song,so why not (and there were merely three).

But it was a simple text layout for a short article. Personally I can’t complain. Hence I ask what I’m missing.

> Honest question: what was the problem?

> But it was a simple text layout for a short article. Personally I can’t complain. Hence I ask what I’m missing.

Just wanted to repeat this as I entirely agree. Sure, it’s bland, but like it’s extremely readable and of all the sites to rant about UX on, this wouldn’t even make my top 1M.

All in all, I find it to be pretty readable. Some bigger body margins would help a lot though.

If you read it on a computer the text floats out the whole window width.

I prefer that to pages that constrain the width and have junk on the sides. I can always adjust the window width for comfortable reading when the author kindly arranges the page that way.

I thought it was refreshing to have no cause to wonder what was going on under the hood. I wish the rest of the web was like that.

It only looks that way because the link to its stylesheet is broken:

The `href` leads to a 404, hence the "refreshing" look...

Given the "Close Form" text and the bullet list of social media icons, it looks like the CSS is failing to load.

https://archive.ph/wOlQm

Must be the first time I've agreed with every point in such a list. Is the company still in business though? Looks like the site is broken in several ways in both Firefox and Chromium.

Haha, none of these is "easier to hire" .. which is really dumb for that js library (the framework claims to be). Our industry is rotten and learns nothing.

Great list. Is like to add theres no such thing as code perfection.

my close friend feels bti360 is the best contractor to work for.

I clicked through to their website and it is hmmm... different? Like I think maybe it was created using Microsoft FrontPage in the mid 90s and never touched again outside of posting new photos of company events. While trying to discern what this company actually _does_ (which is never really stated but seems to be like government security contracts?) I tried clicking on the "Apply Now" button and just got a lot of errors in the JS console.

That being said I think this blog post actually is spot on and really does capture what it takes to do good software engineering.

does css load for anyone else: https://www.bti360.com/careers/

Very good list!

ruby lock-ins?

Solid.

Reply on news.ycombinator.com