I don't know how to build software and you don't either | Hacker News Reader

I don't know how to build software and you don't either

seangoedecke.com

32 pointsgfysfm53 comments

While I appreciate the author's humility, I don't appreciate them putting it on to me. These questions have answers, just not the ones the author is looking for. The answer is the process. You take your context, your objectives, your team, and do what makes sense. And time changes all things, so constantly re-adjust. I prefer statically typed languages, but are you wrong to choose Python if that's what you and your team is most comfortable with? Absolutely not, if your goal is to deliver something using the knowledge you have now. I think the author is looking for a static answer: X is always right. And that isn't reality. It isn't reality in engineering, in art, and in play, all of which software can be done as an activity.

I’d call it faux humility, given the second half of the comment — which is a huge assertion.

Given the fact that practically no one can say with certainty that their software is bug-free, they are right of course. Except for toy problems.

One term that captures a sizeable slice of the discussion is Emergence[1]

There is some size tipping point (waves hands) where the Toy Problem exits the nest and starts encountering the Real World. The really good RDBMS design encounters data at scale that exposes fresh pain points, and so forth.

The better engineers will have a fighting chance of surviving the onslaught.

Gall's Law:

"A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system."[2]

[1] https://en.wikipedia.org/wiki/Emergence

[2] https://en.wikipedia.org/wiki/John_Gall_(author)

There must also be a related law stating that managers will never accept the validity of Gall's law.

"The Law of Very Few Managers are Worth a Sh_t"

Well, it's at least a theory with a fair bit of supporting evidence :-)

That’s not inability, that’s client preference.

Very few clients want to pay for certified defect free software, as they prefer the lesser cost (and greater speed) of defective software that’s patched as defects become problematic. So most software exists without major defects and in a way that repairs are minimally expensive.

Delivering client preference is good engineering.

This. Exactly this.

Software engineering isn't about perfection. It's about software that adequately meets the need, which is a lower standard. And cost and time are part of that "adequately meets the need". Perfection would be wonderful, but if it takes forever and costs infinite money, then we're better off with something we can use and afford now. (The rigorous pursuit of perfection, on large systems, means no software, ever. That's not a net win.)

So there's a local maximum in the value curve, between "worthless because it's too buggy" and "worthless because it's not finished". Hitting that sweet spot - or close - is what software engineering is about - delivering software that's actually usable and useful in the real world.

You can't have that attitude if you want secure software.

Yeah, but for many things, I don't need it. Do I need a secure word processors? (I mean, yes, in some environments I need a word processor that can operate on classified documents and be guaranteed to not make any copies anywhere else, or make any network connections, or leak in any other way. For most of my life, though, I just need a regular word processor.)

Which brings me back to my point. If I have to get a fully, 100% guaranteed secure (and also crash-proof) word processor, I'm going to be waiting a long time, and it's going to be very expensive. Meanwhile, I've got stuff to write today, and my budget isn't any too fat as it is.

Sometimes fully secure (or man-rated, or 9 nines, or fully validated, or whatever) software is what is needed. Much of the time, though, it's a better engineering trade-off to not go that far.

(40 years building software)

I think the most significant aspect of building software that is rarely discussed is the business’s appetite for change.

I can solve any architecture problem, but I can’t lead a horse to water.

By far the biggest hurdle to building good software is gaining buy-in and trust from everyone.

The actual architecture is like skiing a double black diamond. If you’re at that level of skiing, you can ski anywhere.

What is the europe equivalent of double black diamond, ungroomed black?

In Europe, we use black, red, blue and green colors. The rest is off-piste.

The difference between piste and off-piste is that of insurance. You are usually covered by the generic insurance on-piste only.

It is intrinsically difficult for the know-nothing money guys to trust people whose expertise is of a vastly different order -- across various dimensions such as technical difficulty and creative explorative engineering -- than what it takes to make deals on a golf course.

All they know is that the longer it takes us, the more money it costs them. And they don't want to know why, they just want us to wave the magic wand and deliver our systems, even though our best estimates are really just "guess a number, then multiply by three and add five."

Their entire job is to wring money out of the peasants, even when we're well-paid, well-intentioned, and hard-working engineer-wannabes who love what we do. Hell, most people (including us) don't realize that every new significant software project is an sublime artistic endeavor cut from whole cloth with rusty, bent scissors in the dark on a bumpy train ride, under fire.

It doesn't help that many of our peers are just paycheck-earners who only try their best to do the least they can to deliver the barest-minimum of quality, for the sole purpose of getting a better job somewhere else.

I've never looked for a job while I had one. I call that "being honestly committed to the job"; the mediocre of the world (most people) call it "not playing the game".

"So sick of complacence." --RATM

> I've never looked for a job while I had one.

Same, and that keeps biting me on the arse. EVERY TIME.

As for the rest of your comment -- it is sadly spot on. I am extremely tired of complacence myself.

If there is any guideline I follow, it is that any complexity must be justified.

When for a client we have a choice in hosting environment, we still choose Windows because a shared IIS hosting environment or Az App Service in both cases allow deployment though MSDeploy and don't require docker because on Windows you can run many apps side-by-side without issue. This saves a whole lot of hassle.

For frontend, any library or build system needs to be justified. No webpack because it causes more problems than it solves. A simple gulp build suffices.

In terms of software architecture, a simple webapp-database architecture works fine for 80% of the use cases. No need to introduce extra parts with extra complexity. Boring architectures work best. Any complexity can and will be a source of headaches.

I could go on and on.

I agree completely. The simplest solution is often the best solution. And also, the best understood and best supported solution. Teams love new shiny objects and toys to play with, but if business leaders truly understood the tradeoffs, they’d shy away from the latest and “greatest” most of the time.

A better resume is built with SQL Server EE than it is with linked-across-the-LAN Access tables. It's also a better solution, but takes far longer to implement. [True story from nearly three decades ago.]

A selfless engineer doesn't care if Perl gets the job done quickly but doesn't look as good to prospective employers. A selfish person weighs their design decisions with more personal factors, no matter how they affect the entire project.

Still, making functional art that is also beautiful is its own hindrance, so we, too, are not without fault. Unless that artistic process is necessary, but who's to say? Aye, there we walk the razor's edge, my friend.

> ... and don't require docker because on Windows you can run many apps side-by-side without issue

As opposed to Linux which would have issues when running apps side-by-side?

Linux took over the entire world and now run on billions of devices and there has to be a reason for that.

One example of devices running several applications side-by-side that we all use would be smartphones: Android or iOS. No Windows to be seen.

If a part of the world moved to containerization, it's not because OSes had issues running applications side-by-side.

> As opposed to Linux which would have issues when running apps side-by-side?

The meteoric rise of containers is a massive testament to just how bad Linux is at running apps side-by-side. Or for a more principled approach, take a look at Nix. Either way, the problem is solved by adding quite a lot of complexity.

Android and iOS also solve this with a large amount of sandboxing, which means more complexity. Developing and maintaining mobile apps is anything but simple.

To be clear, I'm not saying these things are bad. They are necessary. But they are not simple.

Linux is emphatically no worse than Windows at running things side by side. Better in many places. But socially it has a custom of using a package manager for dependencies, where Windows instead requires you to vendor everything, because historically it's just had no package manager story.

People sometimes criticise ecosystems like npm and Cargo because you end up with many dependencies. But the alternative was C/C++, where you instead ended up vendoring everything, which has its own problems, and reimplemented things poorly because getting dependencies in is so hard, and that's fairly terrible.

I would suggest that actually containerisation was more an opportunity that Linux could avail itself of, since it was a fairly small shift, things were already normally pretty scripted, whereas it has never taken off much on Windows because everything's already too manual, it takes too much effort to shift.

The reality is probably a bit of this, a bit of that, but Linux is not at all bad at running things side-by-side, so long as you either commit to using the OS package manager tools, or deliberately avoid it entirely, like you effectively have to on Windows.

Nobody is arguing that Linux is bad at running multiple processes at the same time. The whole point is the differences in how people customarily package software for each platform.

It turns out that package distributions on Linux are nice if you only consume from the package distribution, but are a huge pain in the ass if you want to deploy software packaged independently (which applies to a large portion of commercial software deployments).

For proof of how fucking hard it is to effectively vendor your software dependencies in Linux see: Docker, Nix, AppImage, Flatpack, Snap.

It's not hard because of any technical limitation of Linux.

Packaging is pretty easy if you want to stick to a single ecosystem. If you're a Red Hat shop and you're only interested in deploying your stuff on RHEL, then rpms are easy to build. Use the system deps where appropriate, ship your own when not.

The packaging story falls apart when you want to ship software to customers running god-knows-what.

> As opposed to Linux which would have issues when running apps side-by-side?

Side-by-side versioned libraries often turn out the problem, which is one of the driving forces behind containerization technology: One version wants version X of that library, one app version Y, and then also every distro has different flavours of the same library.

I like this idea. I think we can dismiss the most literal/extreme interpretation of this (as most commenters seem to be doing), but when I look at most of us I find we are more likely to forecast with overconfidence than underconfidence (tech X will destroy the company, or NOT using x will destroy the company).

I remember working at a place and our whole data-team (myself included) seemed like a useless waste a dozen engineers. I thought the company was adrift. A few years later it was bought for billions of dollars because the business people knew exactly how to threaten a dominant player in the market. And I realized my henny-penny attitude was complete noise, I just was out-of-the-loop about what the long-term play was.

I think a lot of this is that we each want to be the center of our own universe, we want our problems to be the THE problems, and to some degree understand that peddling our advice about tech X or Y is to some degree advancing our own influence.

In the end, this is a question of judgement.

Everyone doing senior work in any field is doing judgement. What are the pros and cons of doing this thing? What are the short and long term consequences?

There's not actually that much we can generalise about judgement itself. Did you think about it, considering all the tradeoffs? Did things work out?

So a lot of the high level advice I see in the Internet is basically just turning the same stones over and over.

I feel like the most useful articles are actually about specific things like how to write a memory allocator. Those are more like sharing experience that can be used as the raw material of making good decisions.

Maybe not me but I feel like someone knows how to build software on this site?

Yup, but I ain't telling y'all sh_t. You can figure that sh_t out for yourself. ;-)

I'm not really impressed with what y'all are building anyway. More FANG-sh_t? Microsoft-adeverywhere-2030? Salesforce? Better insurance software or real estate collusion? More Google artificial-ad-f_ckery? Better social media disinformation bots?

"Know your enemy." --RATM

[And, nothing personal, my friend. I'm addressing the industry via the "Royal You", representing by YC et al.]

you can say shit on here. I'm sure ratm would give you a pass on using that word this time. if not they'll just have to take that power back

I give them all my power, every day. And often shout at the living dead.

I build lightweight communication interfaces that can run off cheap hardware without connection to the wider internet for off the grid setups with vanilla html/css. So not quite.

Yeah, that's why I added the caveat, after giving you an up. I was speaking to the wider audience that frequents these parts.

I'd love to see some links. Is part of your work open-sourced?

I’d like to challenge a core argument of the article:

> Only your last ~20 years of experience matters for these questions, because the basic landscape of software development changes so rapidly. [...]

… with an anecdote. I’ve recently skimmed through a “Thinking Forth” book, and, language-specific information apart, was surprised to read that the software structure they recommend resembles to what I’ve figured out over years of programming by intuition.

The basic landscape "changes" so rapidly because although we keep talking in circles, about the same things over and over, for some reason we give them new names each time we go around.

Apparently, yes. I remember reading a comment stating that with Redux, Facebook rediscovered UI optimisations of Windows 95.

Yeah he is plain wrong on that.

I started to to code 25 years ago when I was 12 years old, creating chatbots to manage RPG sessions for a IRC server. Turns out that's the most up-to-date experience I have now with the advent of LLMs.

This is nihilistic nonsense. The author's problem is that he's only ever seemingly worked on web stuff. People stay in their domain far too often and then come up with big statements like "I have 20 years of experience and don't know what I'm doing." Is that maybe because you stick around a domain and layer defined by people with an average 2 years of experience, many of whom learned their job from a Youtube tutorial?

It's possible for organizations to get better, even good at building software. The foundations of the field haven't changed much, people just don't learn about them and go on to build towers of overwrought abstraction, which is the thing that keeps constantly changing.

If you think of React and Redux as foundational, then everything you say has water under it. Go open a TCP socket.

Rant accepted :-)

One thing that certainly has changed, over these years, is that the size of applications is much, much larger now.

And, certainly, we never had to think about security in the 90's, for regular-degular business apps, anyway. Now, the security dimension is it own barrel of worms, both affecting our software design and requiring network security specialists as well, configuring our boxen. I doubt very many of us are working on pure intranet apps.

So, yeah, the roots are the same, but the trunk is much larger, higher and much more expansively bushy, and its environment is considerably more dangerous.

It's definitely harder, as you say. Not only do you have to think about technical aspects of security and privacy, but also the legal requirements for those things, and how they differ across countries.

But the complexity is not so much worse than it was in 2016, and yet in 2016 we could manage it.

The type of things I see in the FAANG world lately would not be happening 10 years ago: CI that randomly fails 5% of the time because everybody knows React, but nobody knows how to set up a Linux machine. The fact that USB drivers on M1 macs are still broken and nobody at Apple knows how to fix them, or that Apple seems to have no one on staff who knows what EDID is.

We're not failing because of new complexity, it's the stuff we put up to manage the old complexity that we can no longer service. And that won't get better until we accept that the answer is not "rebuild it from scratch, but this time let's have new grads do it with no training."

> I doubt very many of us are working on pure intranet apps.

I think this is part of the problem. The reason why your Mac will kernel panic if you plug in two external screens from the same production run is because nobody is working on "purely intranet apps" like the display driver.

I don't want to sound like I believe we were somehow smarter 10 years ago. But you had relatively solid foundations to build on and people were incentivized and given mentorship to attain mastery. We need to bring that back.

Yes to all that, but the blame -- as always -- belongs to the money people. They are the ones chasing shiny new sh_t instead of fixing what their existing systems are f_cking up royally. It bears remembering that those money folks are the ones choosing the CTOs, which is less likely to do with their technical solidity, but more likely to be because they will fall in line. But that's just my educated guess, by understanding money-centric folks and how they almost always operate. The love of money has corrupted and is corrupting so many branches of modern life.

> t bears remembering that those money folks are the ones choosing the CTOs, which is less likely to do with their technical solidity, but more likely to be because they will fall in line.

Wish somebody told me that 25 years ago. I woke up to this reality way too late in my life and career. Now I started getting that it's about emitting the right signals: mostly that you are meek and malleable, if you want to get hired at certain places.

I of course refused to do that, many times, to the detriment of my career. Though maybe it was just shitty luck 7-8 times in a row, who knows. Also Eastern Europe is far from a good environment, so...

Same here, but I never sold my soul to them, so, while not having nearly as much money, I have my self respect. I hope you feel good about fighting the good fight, even if you did so by remaining in your natural, more innocent, state.

Peace be with you.

I don't like the assumption of the author you must have observed these architectures in action. Nor does he think you can learn from a 4 year old architecture that was in place when you joined the company. Nor can you read books and papers published about companies' architectures.

He touches on this point only to reinforce that you can only learn by experience, which is at least partially incorrect, even if I agree its a fantastic teacher.

It's a nice bit in humility and a reminder of the length of time it takes to reveal major effects of architecture decisions but it paints a bit of a false hopelessness.

This reminds me a lot of my feelings for Wayland color management being ready to ship color management. https://news.ycombinator.com/item?id=42284035

A lot of software does take a lot of trying & stumbling through to get good.

Alas many companies simply allow the rubble to build up around them, are on to new features, rather than figuring out how better to stitch their systems together after the first bad pass or two. Sure, X11 was there... it wasn't good. Trying to be more than haphazard takes work, and time.

I don't like the assumption of the author you must have observed these architectures in action. Nor does he think you can learn from a 4 year old architecture that was already in place when you joined the company. Nor can you read books and papers about lessons learned published by other companies.

It's a nice reminder to be humble and at the length it takes to see major effects from architecture decisions but he paints a kind of hopeless picture that just isn't there.

The author argues against absolutism in SWE, but makes absolute claims about experience requirements to understand what he’s talking about.

These pseudo humble tirades are getting tiring. Just grow a backbone and own decisions that you make, Jesus.

And I do know how to build software. Whatever narrow definition I have of it.

And no, this is not coming out of defensiveness but frustration.

This is why consulting is such an awesome career path once you have some experience under your belt. It is absolutely possible to be well-informed enough to answer these questions, because consultants see the beginning, middle and end of these decisions!

All of these calls about which way to build software are judgment calls.

For example - a monolith has strong benefits but when you reach a certain size & team size it makes sense to split out some things - but you should still keep the monolith where it makes sense. And redux state storage makes sense for 99% of cases but the org should have flexibility to let teams run things their own way if they can make their case, depending what controls your org wants to have in place to keep a certain level of standardization. It’s about finding a balance.

The author seeks universal answers where none exist. All these different approaches have survived well enough to enter the zeitgeist so they each must have merit. The fact they’re contradictory suggests the appropriate mental model is more like a flowchart with many end states, than a path that seeks a holy grail.

I also strongly counter the author’s assertion that 20 years is not enough time to see all this play out. Of my 20 years’ software dev experience, 5 were at a company that went from less than 2000 employees to over 40000 while I was there. I saw the architecture evolve to meet the challenges of growth through many orders of magnitude of scale, and saw the hiring of people who’d done the thing in other companies at the level we needed and brought the leadership to implement it within our teams. 20 years is absolutely enough time to get in one of these 5 year experiences even if the other 15 years is throwaway. To maximize your chances, go and work in SF or Silicon Valley. Yes, it does happen elsewhere in the world; but with far less regularity. If you really want to hone your craft you have to go to the place it’s happening.

This is a dumb post. I can't believe someone with 20 years is still looking for X is better than Y, black-and-white answers. All these questions have answers, but they're not blanket ones. You have to get into the details. But even without getting into details, I could probably ask ChatGPT and with a short conversation get to the crux of each matter. Most of them come up frequently enough on HN to have stock answers that the author doesn't seem to know about.

Microservices? It's about parallelizing and scaling teams. By decoupling codebases, tech stacks, deployments (to a large degree), you lean into Conway's law and reap benefits with other costs like dealing with eventually consistent behaviour.

Typed vs Untyped languages? Answer is use the language you know when starting out. A lot can be done with either, in most cases it comes down to preference and adeptness of what you know. For large-scale standard software (e.g. database, API, front-ends) using a statically-typed language will allow larger groups of developers to work on the same codebase with fewer surprises (like a typo in an unexercised codepath). But the sunk-cost is not a fallacy (or is very high so tread carefully), you can't stop and rewrite your entire business in Rust and compete to survive.

Blah blah blah...

Even if you work at each company for several years at a time, if you're paying attention you can see that a thing they did many years ago is tech debt on current development and operation. You don't have to learn in real-time, learn past history of the codebase you're working in. If you only work at early startups, try something different...

Like what some other comments are saying, try different things, expand your horizons, gain a wider perspective than what folks who are doing exactly what you're doing are talking about. Most of this focus on code and packaging is plumbing as far as I'm concerned. The actual thing software does is transform data from one thing to another: datastructures, algorithms. A higher level view, databases and SQL. The other stuff is a moderate puzzle of filling in the blanks.

Stop trying to find answers by appealing to authorities, f#@*-around and find out.

Edit: I actually entered this into GPT-4o and got expected results.

"The following is a post on Hacker News. I want you to look at the examples of unknown things given, and for each one, get to the crux of the matter providing a concise why/why-not, when/when-not, pros/cons.

..."

Specifically for the Expressiveness of different languages (including static- or dynamic-ally typed) I use this post[0] as a counterexample to any dogma.

[0] http://thume.ca/2019/04/29/comparing-compilers-in-rust-haske...

Reply on news.ycombinator.com