Software engineering is a learning process, working code is a side effect | Hacker News Reader

Software engineering is a learning process, working code is a side effect

lambdabytes.io

237 pointsio_nathan74 comments

Reminds me of this article by Li Hongyi: https://www.csc.gov.sg/articles/how-to-build-good-software

Quotes:

"The main value in software is not the code produced, but the knowledge accumulated by the people who produced it."

"This is why relying on external vendors for your core software development is difficult. You may get a running system and its code, but the invaluable knowledge of how it is built and what design choices were made leaves your organisation."

And from the first quote, which I strongly agree with, it follows that when people leave the company after building something together, the organization is losing something invaluable. I am baffled that this is not understood and I have witnessed first hand how great teams that are able to work well together collapse and the company has to rebuild the knowledge, wasting time and money.

I suspect most of the time that's because you can't justify keeping a team idle for even a week between projects if that's what it takes to ready the next project for them to work on. I'd be surprised if most businesses even allowed a day of idleness to get a running start with a well-functioning team.

Spot on. The tragic irony is that giving a good team a breather will usually pay dividends, from a strictly utilitarian ROI perspective.

Which is why the concept of 20% time is important. It means that 1 day a week your manager can't tell you what to do, so you can work on whatever you think is most important. You could use that time to work on what your manager wants, and most do, but if you have things that you feel are more important you just go and do them.

I had a friend who worked as a project manager for a major telecom and their policy was to basically furlough her for a few weeks (paid time off) after a major project winds down. She's there if something bad happens, but otherwise it's downtime (and retained experience!) before the next project. It seems like a sensible and healthy policy.

Perhaps it is different where you are from, but I would usually understand furlough to mean unpaid time off at the company’s request.

I just meant it in the general sense of the word, as a leave of absence at the company's behest that could be paid or unpaid.

That quote's perspective now seems to me to be correct and fundamental to our industry, but it was basically ignored during my degree and I'd say it's still very much under-appreciated in software engineering.

I'd guess it looks under-appreciated on the surface, because it is hard to run a business on such a premise.

Admitting it "officially" ultimately boils down to retaining by all means the engineers who already acquired the knowledge. Some of those will ask for more benefits, others will play games of power, others still will simply refuse to stay once they stopped learning.

(In my industry and geo) businesses chose the opposite -- they slice and dice the knowledge into tiny bins so that a total newcomer can learn up quickly enough. This drives down wages but now requires many more people where a few should suffice.

Solution to that? Total subcontracting of sw development. Which is of course a slow death for the business, but a) at least it looks like a business model, b) death is slow enough so that managers have time to move on, c) this has a merit of being implementable. The alternative -- finding people who already know or learn very fast, and then keeping them -- is not for every budget.

> Admitting it "officially" ultimately boils down to retaining by all means the engineers who already acquired the knowledge

I mostly agree. Admitting it acknowledges the importance of the humans in the system, and diminishes somewhat the allure of the code they write. It pushes for more focus on learning and sharing, and strengthened the labour power of the human interacting with the engineering system, as they're full acknowledged as inseparable from it.

Speaking purely of the headline... imagine saying that in a client meeting: your own learning is the real gem, the working code is a potential side effect that the client may benefit from. Imagine saying it with a fully straight face, deadpan, and not backpedaling on it.

Clients are worried sick that their businesses won’t exist unless they make the right key technology investments. They are worried they will be looking for a job if their contractors can’t deliver. But as long as the contractors learn enough, it’s all good.

Again, speaking specifically of the deliberately provocative headline

I had initial thoughts, but I read the article and noticed the author is an academic.

The author mixes up How and Why of software engineering. Learning is part of the How of software engineering. The Why of software engineering is to develop and deliver solutions to business clients. Most business client don't care if you learn something or not, they expect results.

True, I am an academic (btw is that something to be ashamed of?), but I have worked 10 years in the industry - and am still doing projects for real clients.

I agree, I could have differentiated a bit more on the How and Why, but I think in the end it applies to both: do the customers REALLY always know what they want? They think so, but it is not really the case. Often it is muddled under a premature idea of a solution the client already thought up. Therefore, its better to start with a minimum viable solution (prototype) so the client actually sees and touches something tangible, which will very often change their idea of what they actually REALLY want.

Strongly disagree. Why of software engineering is to formalise business processes so they can be replicated and automated. The code is most definitely "how" we get there.

Software development is a knowledge business, as Fred Brooks has realised nearly half a century ago [0]. In order to build any kind of working software, you have to learn a lot of different things first, on all levels -- from how to write a list comprehension in Python all the way to why a certain end-user needs a particular functionality.

[0] https://en.wikipedia.org/wiki/Brooks%27s_law

I feel that you definitely could say this of art like drawing or writing, but not software. Programming is a tool I use to meet an end. I decide I want to have this program, spend a week writing it, and have the program. That's using the skills I already have. I don't care how much I improve, because to achieve the result I wanted I didn't need to hit the books and practice for a month, improve, and come back again.

Maybe that's why I find it so hard to get into art.

Why can't art be the same?

I used to say "I can't draw", then I sat down and spent about five hours looking at how to draw books, working through the basics, over the course of about three weeks.

Now I can draw, repeatedly, to a certain standard. If I want to learn more I could, but it isn't strictly necessary.

> I decide I want to have this program, spend a week writing it, and have the program.

How do you decide you want 'this program'? Why not a slightly different program, or the same functionality implemented with a different approach. It boils down to research and learning, especially for non-cookie cutter projects.

In my experience, it works out better to know the truth and act in accord with it, however inconveniently it contrasts with how you would prefer things. The client probably should realize the hubris of the endeavor and expect failure.

I expect this is said in client meetings all the time. You hate to be a lowest-bid commodity supplier; you want to play up your long-term high-touch relationship and understanding of the client's business. "The code for this application is just one of many tools along the way to our enabling your digital transformation strategy."

I've never heard this in any client meeting and suspect most clients wouldn't take this very well.

Clients are interested in a working product, as quickly as possible and as cheaply as possible. It's hard enough already to convince them of the importance of non functionals such as reliability, maintainability and security.

The headline makes sense to me, but:

> It is no surprise that agile processes have been proven to be so successful and useful: they emphasise collective learning.

What? Every place I’ve seen “agile” practices ended up far overemphasizing short term thinking over any sort of respect for learning and the collective knowledge of the employees of the company. I’m not saying the two are necessarily at odds, but the sort of management that is attracted to agile is also the sort of management that probably doesn’t give two licks about whether you’re learning anything deep, so long as you get the story points closed.

I agree, Agile is often used as an excuse to not plan anything - which is obviously unprofessional, completely wrong and leads to disaster. I think this was realised by the Agile movement as well, and they have come up with various techniques to also keep track of the long-term aspects e.g. Impact Mapping, User Story Mapping, maybe Event Storming.

Automotive mechanics is a learning process, fixing cars is a side effect.

Practicing medicine is a learning process, healing people is a side effect.

See how ridiculous this sounds?

Not really when you consider that learning is always a prerequisite to the examples you listed.

You can't fix a Ferrari without learning about a car's history and the previous owner's maintenance to trouble shoot.

You can't fix a patient without learning about their history and learning about interactions with the treatment you prescribe.

I agree. But I'm on board with a similar take like: the net present value of a software engineering effort is much larger than the NPV of the end product, because the learning unlocks so much more production down the road.

If every car were custom made, you'd have no idea which control had what function... look at how differently a mass produced car used to work before things were standardized https://www.hagerty.com/media/car-profiles/driving-a-model-t...

You can't fix something you don't understand, and unique things require a lot of time to figure out.

This applies in many areas. I learned a ton about Atomic Clocks the first time I fixed one... the second time I did it, it was only an afternoon, instead of a week spent in my friend's shop over the course of a month.

So, yes... the main investment is the knowledge learned. It is foolish to waste that effort.

Your friend’s shop has an atomic clock?

At one point we had 3 of them working, and 3 rubidium clocks and 2 GPS disciplined 10 MHz oscillators... it was wild.

In theory, we could have run one upstairs, and one downstairs to see if we could check relativity... but the tubes have an finite lifespan, so it wasn't worth it.

The Cesium clocks have since been sold... and the memories remain.

Would love to learn everything about the clock. Why did you have it in the first place?

Not really. If you are running a car repair shop or hospital, it’s difficult to derive ongoing value from the cars/people you fixed up in the past. The tricky thing is that code shops hold on to their code and often code shop managers vastly over-value that code and under-value the domain experience built up in their employees while fixing up that code.

Wow, this is a magic template for quotes.

"Life is a learning process, happiness is a side effect."

I can't agree more ;)

That's because neither of these things you mention are acts of constructing something out of the void. Something unique that will have to be maintained afterwards. Software Engineering has it's own challenges.

> Automotive mechanics is a learning process, fixing cars is a side effect.

Well, no. but replace automotive engineering - a more direct analogy to software developer - and yes.

> Practicing medicine is a learning process, healing people is a side effect.

As someone married to a physician, you'd actually be surprised how accurate that statement is. There are many thinks we have medical confidence, but there are also many things that we have limited knowledge about - especially new areas of medicine.

----

I don't think your argument sounds that ridiculous. We know a lot about computers. What we often don't know a lot about is the business requirements - which are fundamentally human driven. In my experience, technical knowledge is rarely the limitation for good software. It's an inability to discover the human needs of someone else.

I think that's why they calling it "practicing" medicine, instead of human engineering.

> Practicing medicine is a learning process, healing people is a side effect.

> See how ridiculous this sounds?

That sounds perfectly fine to me. I very much profit from the fact that humanity has been healing people for centuries, because it means that treatments, hospitals, and a whole education system to educate doctors and nurses exist.

The fact that some John Doe survived his surgery in 1923 and made a recovery doesn't help me much at all.

Sounds ridiculous, actually. /j

Fixing car mechanics is a known process - you got calibrated expectations, developing a new algorithm or drug is an unknown, so it can be described as a learning process. It's search for new values as opposed to exploiting known values. What we are learning is the concrete process at hand and its affordances.

Truly is. I think egotistical maniacs are the biggest problem in tech.

A big problem is capturing what your team learned, as semi-structured documentation. Otherwise tribal knowledge fades with time and turnover, and you’re finally left with a system you don’t understand but luckily still works.

If you're lucky enough to have frequent bugs bam! spaced repetition.

It has to be something that the team commits to maintaining. IME, often there are a couple of folks in the entire team who contribute with any regularity, or care to update the docs. Its often not an activity that's given credit. When the people who maintained the docs switch teams, there is rot- docs become stale. Because they're stale, other users don't trust them don't update them, just stop using them leading to a vicious cycle where they become totally irrelevant.

True, but I would say that if (thats a BIG if) the code is written to reflect the domain (see Domain-Driven Design), then other developers should be able to pick up the main concepts of the domain pretty quickly. Maybe you have some artefacts leftover from some Event Stomring sessions or a User Story Map or an Impact Map,... all these come in handy.

> Even worse, the idea that SE is a learning process is not communicated in any of the SE textbooks I know.

I think that Naur's "Programming as Theory Building" (1985) communicates this idea succinctly and forcefully. It's not a textbook, though.

Also https://www.amazon.com/Software-Capital-Economic-Perspective...

Naur's emphasis on "personal advice" reminds me of Bloom's 2 sigma effect https://wikipedia.org/wiki/Bloom%27s_2_sigma_problem

  "the average tutored student was above 98% of the students in the control class"

I guess "personal advice" is teaching the code's theory, and much better than learning from a one-off textbook (i.e. documentation).

This is only true because of abysmal quality of bookkeeping. Choices and conditions they were made in are not documented in non-ephemeral form - often email, word of mouth and even if at all, in strongly lossy compressed way.

The more lightweight you go, the more information is lost. You can easily use hardware made in 60s. Trying to use some software today made half a year ago can be a journey. (Algorithms are another matter, they do sometimes survive if they are well described.)

Learning is involved in anything but it is not the process of making software. It is not the process of anything, as it cannot be put in a neat box for any management. A process is at most output of it.

I get the general point of the article, but if I just lost the source code to the app I've worked on for years… It would take me a very, very long time to re-write everything. Sometimes, getting relatively straightforward code to actually work is hard and time consuming.

Indeed - but its not that side effects are something we do not want in programming: they are generally speaking the very essence during program execution ;)

Other current engineery thread here: https://news.ycombinator.com/item?id=25823907

I think one can pursue software engineering with different goals in mind.

Maybe you work on a side project with new technologies to learn things - in this case the working code is not as important.

Perhaps you are trying to push a product out quickly - working code is the end goal and you may learn a few things along the way.

I find it disingenuous to say that working code is merely a side effect. As the author says, SE is a learning process and the main challenge is to understand the domain. However how the outputs of the process are ranked in importance is subjective and therefore one can rate working code higher than learning.

Although if you consistently rank working code over learning I would assume you are a poor software engineer.

I think the overall point of this article is that SE is a subject that is best learnt by doing, not that engineers do not need to work with end product in mind. So the title is slightly misleading.

I bet someone who makes a living selling working code would disagree.

I make a living selling working code, but I agree. I think that, just like how monitoring needs to run at higher priority than actual production workloads, the understanding of programmers needs to be at a higher level than the understanding which is actually encoded in the actual production program.

I agree as well, and the real test of understanding comes if I forget to create a backup and lose my program.

Rewriting the same thing from memory when I already know how to structure the program is boring but fast.

There's no such thing as a working system.

That's why space stuff has redundancies etc.

one can argue that redundancies is part of a working system, in fact most large distributed systems are designed with redundancies.

Professional software engineers develop and sell solutions. Code by itself is meaningless, it's the application of code in certain parameters and delivering expected results, which results in solutions.

- There's no such thing as programming.

- There's only research-programming.

I’d argue that research is to try to pass knowledge learned to someone else ( which is why professors teach and do the majority of research ). The first thing that goes when a deadline is coming up is... documentation... so I’d argue no, even a cobol program in a bank who no one knows about but is 100% stable has value.

This quote is why software engineering isn't engineering. Civil engineer: "Civil engineering is a learning process; safe economical structures are just a side effect."

My understanding is that civil engineering works that way too; the clients are just more willing to pay for enormous safety-factors.

the quote is from Alberto Brandolini, inventor of EventStorming; the context is that working code proves one has learned at least one functioning model

as a practical note, the only development artifact guaranteed to survive the lifetime of the product is the code

This is a meaningless collection of buzzwords. I'm looking forward to see the author write object-oriented Haskell.

Woah dude, so deep! *rips bong*

Software engineering is empowering managers, working code is a side effect.

Working code is a side effect of being a Software Engineer.

Amen!

Agile is lightweight?

I agree with this statement. No matter how smart or experienced you are every project is a novelty unless you are copying and pasting. Learning is the lowest common denominator in this field.

The author seems like an academic and doesn't seem to have practical engineering knowledge.

Engineering is the process for solving problems, usually at business scale.

Mechanical Engineering solves problems of moving widget A into widget B, at scale of thousands or millions of widgets.

Electrical Engineering solves problems of moving electrical power form point A to point B, at scale of generating and delivering electrical power to millions of customers.

Software Engineering solves problems of moving digital data from point A to point B, at scale of collecting Billions of user data into databases and generating metrics reports.

And how do you apply? Even when you have premade solution (from somebody else), you have to learn it. Most of the time you have to learn how to solve a problem not familiar for you.

The same applies to other industries. How do you imagine smart grid does appear?

There are differences in How and Why of software engineering. The author attempts to answer Why of software engineering by answering some aspect of How of software engineering.

Learning is part of the How of software engineering. The Why of software engineering is to develop and deliver solutions to business problems.

Reply on news.ycombinator.com