Ask HN: When teaching myself, how can I learn to engineer instead of just code?

Software engineering knowledge is similar to other types of knowledge, like Java or Node.js programming knowledge. It is available on the internet.

There seem to be quite a few people who still think that software engineering is about planning, creating design documents or sequence diagrams, building the ultimate design, etc. Although planning and design are important, this general viewpoint is outdated.

I believe the the keys to software engineering, rather than coming up with the ultimate plan or design ahead of time, are A) separation of concerns, and B) tight, closed, feedback loops with incremental improvement. Your tools and processes are going to embody the core engineering principles.

As far as separation of concerns, there are many ways to do that well and to do that poorly. But I believe that currently Node.js with its simple, one-purpose per module system with semantic versioning gives you the best head start in that direction.

Another core feedback loop for contemporary software engineering is automated testing: unit tests, integration tests. Unit tests keep the loop tight.

Continuous integration is also key and can be thought of as a way of making sure you have a closed loop because it means that all of your tests _have_ to pass before you deploy which means that you must immediately make corrections when you have failing tests.

Another critical aspect of contemporary software engineering is quickly incorporating feedback from users and other stakeholders.

So short iterations are very important. However, you can't just take a few pieces of this whole thing that you like and ignore the rest. If you do short iterations without good testing, you will be screwed. You need the whole process to fit together. That is one way that a lot of 'agile' teams go wrong-- the manager thinks 'agile' just means he can release more often using basically the same process and tools he was using before.

I believe that the most important aspects of software engineering are the most recent developments in tools and processes and these are often not yet incorporated into many university courses or majors in software engineering.

So anyway, just google for the names of the popular build, test, and CI tools are for your platform and also learn about whatever abstractions and component systems are available for your platform. Then you have to try to use that stuff to learn it.

I'm currently writing an article about this but the two most helpful things I've found are:

1. Be thoughtful and patient: Instead of writing down whatever comes to my mind, I try to take a couple of minutes thinking about the ramifications of what I'm doing before writing the actual code. In some cases I'll mock several ways of doing the same thing and try to get a "feel" from them [0].

2. Know and use your tools! Unit testing, debuggers or code analyzers are there to help you. You don't need to use all the tools all the time but by becoming proficient with them you'll make your life easier and you can get a deeper understanding of what you're doing.

[0] My "feel" might not be your "feel", it's probably not even the right way of doing things in many cases, but with enough time programming you tend to recognize what "feels" good and what doesn't.

To put it short, understand the limitations of what you're trying to achieve, and find out what you can afford to add and what you can skimp on a little without notice or, to add performance elsewhere.

Without limitations you're just coding, fiddling, dreaming..

If you want to engineer something you should learn about software design pattern. The wiki page is a good start: http://en.wikipedia.org/wiki/Software_design_pattern.

Run your code in Production. Scaling. Robustness. Error Handling. Correct logging. These will all keep you up at night until your code is correctly engineered.

1. Create something.

2. Recognize its failings.

3. Fix that thing, or create something new, without those failings.

4. Repeat indefinitely.

Engineering is a bit of an art form in my opinion. It is about style and how you tackle a problem. I recommend reading Hacker and Painters.