Reflections on a Decade of Coding | Hacker News Reader

Reflections on a Decade of Coding

scattered-thoughts.net

80 pointstim_sw31 comments

There is something magical about the 10 years mark that compels people to write blog post showering us with their wisdom. I crossed that line several years back and my biggest realization is how little I knew back then.

I believe it's worth remembering that years of experience is at best a weak predictor of competency. Careers are non linear and someone might growth the same in 10 or 20 years than others in 2 or 3.

For the best deconstruction of career growth I've seen around (and one I personally identify a lot with) I recommend this blog post [1] from Carlos Arguelles.

[1] https://medium.com/geekculture/career-growth-speed-deconstru...

Yes and no, I think decreasing returns is a real thing. 10 years a long time, and you can explore whatever cave you started out in in that time. People regularly go from high school to PhD in an obscure specialty in 10 years.

However 10 years is also enough time to explore the depth of one cave along with the openings of 5 other caves. That T-shaped skill set really gets wider in the next 10 years, at least it did for me.

The width is also what tells you what you haven't looked at. For me it's Haskell and LISP. Not sure if I've got the monads to go down that route, it might overturn my existing long learned wisdom, something that could be painful with 20 years of experience.

I’ve been coding for over 25 years and I’ve gone through peaks and plateaus. I even ran a successful dev agency 15 years ago, and my coding now is much better now than it was then.

I have also been coding for like 25-30 years, and I definitely continue to get better. I also, though, do continue to get less overall productive... not because my mind is going, but because I am distracted by some new pain in my aging body or am spending more of my time being sad about things that aren't code (these past few years were just brutal).

Inspiring :)

Sure, it is a time period that causes people to stop and reflect. I'd wager more people reflect on their lives at 10/15/20/25 years after hs/college graduation than their 11/16/21/26 years after

> I crossed that line several years back and my biggest realization is how little I knew back then.

Mine was that at this mark I already started forgetting stuff which I knew by heart as a junior dev.

I've discovered that adding unit tests to an existing code base doesn't work very well, as each piece of code tends to have too many interdependencies with others. The code has to be organized from the ground up to be compatible with unit tests.

I find unit tests to be a good fit for 10% of code.

Most of the time, bugs are in the glue.

If you have layers that look like: view -> controller -> application services -> model/db, I focus most of my automated tests on everything at the application services.

Critically, I don't mock our db, instead I focus on making setup and teardown fast.

It seems to get me the most bang for my buck.

Then I have a few end-to-end tests for more critical flows.

I find that most of my development time in my career is spent getting that glue in place! Seems like we need a better way to glue things together and make sure that glue doesn't make a mess of things. Maybe AI can help with this

There are tests that help ensure that things are glued correctly. Functional testing is useful in this regard, and end-to-end (sometimes called integration testing*) is extremely useful in making sure critical parts of the system work.

For example, I always try to write a "log in" test, and for systems that take payment, a "payment acceptance" test. Even if other parts of the system have bugs, I want to be able to have existing clients log in, and new clients sign up, no matter what.

* though I think integration testing is really "anything that tests more than one system" without necessarily being fully end to end

Not enough companies use these end-to-end tests you describe!

Which is sad, because well chosen / planned integration tests also can have high code coverage, but no matter how good your unit tests are they will never replace the need for an integration test. Imagine if someone test every component of a car but didn't do a routine road test, right?

I put them in place everywhere I go, so I'm trying my best! I think in a lot of cases the value is difficult to see unless you maintain them as a priority - they can get out of date and start randomly failing, and that's when they go "the way of the dodo" in most organizations, even though a few hours a month of engineering time deliver significant, useful results for your investment.

That's what stricter types are for. And BDD style feature tests, which I find vastly more useful than either Unit tests or E2E tests. Unit tests have their place - in maybe 10% of the code :)

E2E tests are also useful for a smaller purpose - mostly as happy-path sanity checks rather than in guiding development. IMHO.

There are lots of good techniques for adding unit tests to an existing codebase that does not have unit tests without reorganizing everything from the ground up. I like Michael Feathers book, "Working Effectively with Legacy Code" - his definition of legacy code is simply code that does not have tests, so the whole book is more or less about how to solve this problem.

One technique he references that I find very effective is building "firewalls" between different parts of the codebase, and then unit testing those firewalls thoroughly. This means that you can isolate a bug to one part of the system or another, and drastically reduces the surface area to look at when trying to find why there is a bug or why a test is failing.

Edit: In fact, I've seen places in your D code that use similar techniques already - maybe just giving them a name makes a big difference?

I haven't read his work, but the firewalls sound like encapsulating things to reduce interdependencies, which indeed is what makes unittests feasible.

The data structures that go in and out of the modules then get replaced by "mock" data structures.

In my personal, subjective and very non-prescritive experience, unit tests aren't very useful for much. Behavioral Driven Development (i.e. Cucumber tests) at the module level are what I start with when attempting to corral musty code.

I totally agree that, even then, wrangling ungroomed interdependencies are definitely the biggest headache. But I have yet to adopt this approach and not think to myself, "why didn't I start this sooner?"

I've had much success with it when I designed a program from the ground up to do unittests, i.e. the Warp C preprocessor program:

https://github.com/DigitalMars/dmpp

Once it was all put together, nearly all the bugs were misunderstandings on my part, rather than detail bugs.

> In my personal, subjective and very non-prescritive experience, unit tests aren't very useful for much.

Why is that? In more functional oriented languages, they are great because they allow you to verify behavior of functions. Then in the broader application, you're just composing pieces that have already been validated, and the compositions can also be tested. If you're able to incorporate property-based testing, then that's even better.

* Because I care about function, not implementation

* Writing tests has a cost, so I want to right as few as possible while ensuring functionality.

* rewriting tests has a cost. When I start writing, I know what functionality I want, but I may iterate significantly on implementation. This makes unit tests both an impediment, and providing me no safety to perform the refactor/reimplementation.

* the module, or it's equivalent structure in your language, is the optimal tradeoff between standalone functionality. Ideally, a module encompasses a small enough behaviour to fit in one's head. IMHO, an ideal single function should be small enough that it's behavior is readably obvious.

* module inputs and dependencies are easier to instrument in tests. I/O and hardware API dependencies can be injected/overridden. On the other hand, private variables and internal instantiations can be complex and hard to mock appropriately when unit testing, unless you are able to structure absolutely everything as a pure function - IMHO a platonic ideal, particularly in the workplace.

* Functional tests are easier for other coders (and even non-coders, in regards to feature files) to understand. "I wrote 100 unit tests" provides less information regarding actual functionality and productivity than "this feature's scenarios are all passing now."

The originators of unit tests and extreme programming explicitly stated that unit tests are not to be checked into source control, and deleted instead. They felt that unit tests would overly constrain subsequent programmers, and that higher level suites were more useful at providing refactoring safety. The only time to submit a smaller sized test is as part of a bug fix, where one would write a failing test, fix the code and check it in.

Fwiw, while I agree with them on this, I'm don't want to come across as appealing to authority. A lot of Agile/eXtreme/Scrum ideas reflect a simpler and very different time in coding, when code was, by necessity simpler and easier to reason about, and the culture of computers gave engineers much more power to push back on management demands and prioritize code quality. And some of the ideas are just hokey - like eXtreme programmings dislike for branches - encouraging all developers on a team to land their code on 'main' at the end of every day.

Besides, the very first Agile project -that birthed the movement - went over time and over budget :)

With this clarification, I'm not sure I'm understanding the distinction between a unit test and a functional test, so it's possible we're thinking the same thing. And I'm not sure there is a distinction with the languages I use, which are primarily functional languages. To me, units are functions and features that are exported by modules. With languages with immutable data and pure functions, tests are almost needed less at higher levels, because there you have easily understood compositions of things that already work.

I agree that tests have costs, but they are part of the implementation. Just throwing them out means that the implementation is only partially there. It's just a fact of nature for code. One does have to be careful to avoid doing tests just for warm fuzzies though.

I find this is always bean counting... in fact imo most unit testing should be about module boundaries, and not every single function (but fully admitted, that picture changes a bit with functional languages), because that is what is most reasonable. Whoever said that unit testing is about testing your most smallest bits only and you need to test your smallest program decompositions?? (And by that logic, why not test each individual line, or each individual expression???) Because composition also counts and you anyway need to test it, but instead hordes of programmers write unit test with bad mocks that only test unnecessary implementation detail and get fully in the way of any reasonable refactoring, but composition, functional behaviour is not tested.

So imo it is all about definition of what is a unit: It can be your one complicated algorithm heavy function, it can be a whole module at its module interface, it can even be x modules together achieving together a functionality/behaviour.

I can only roll eyes at people who try to tell other people what a unit test is and what not, and what is "wrong" unit testing.. I would hope we'd more distinguish bad vs good testing instead discussing about what is unit and what is integration testing... and most of the time bad tests stem from the crowd wanting to tell what is a unit test and what is not, tbh.

For me this usually means that the code needs to be refactored to make chunks easier to test, or an integration/e2e test is needed, sometimes with the world running in containers (extreme cases when there’s no access to dependency source).

> It's really hard to test non-determistic code. I'm willing to eat a lot of pain at design time to avoid doing so.

Ironically, this is a lot easier if you unit test the non-deterministic parts. Once you can be sure it's easily testable[1], it also has the side effects of being easier to debug, read, more reliable (in terms of branch behaviors), and more readable. Writing longer functions (his leaning) makes these issues worse, so it's not surprising it's a pain point.

[1] Granted, this depends on language. Each language has different language features that make code more or less testable, from a unit testing perspective.

I have to say it, the author is probably great at programming, but poor at writing.

Reads more like a rant, diary with random analogies.

Too much noise in explaining their experiences and forethoughts which aren’t really helpful to the reader.

Perspective can be explained in one sentence and then move on to core pieces.

Apologies but get to the point.

Had to use ChatGPT to summarize the main points for one:

On Bad Advice

The programmer who wrote the post is cautioning other programmers about the dangers of taking advice from online sources without considering the context and details of their specific situation. They explain that much of the advice they have received over the years has been actively harmful to their programming abilities. They suggest that programming practices are mostly tacit knowledge, which is difficult to share and can vary depending on specific situations. The post argues that it is essential to consider the context and details of one's specific situation when seeking programming advice and not to rely too heavily on generalized advice.

His writing is subtle and nuanced which is ironically probably why it lacks what you appear to be seeking, the "cliff notes", ai-summarized version of the article.

Wish i could read it in a pdf-like format

LOL. Guy just wants to let his nuts hang after "a decade of coding"...I knew a molecular chemist ex Canadian Air Force pilot that taught himself Russian on a train ride and was a premier Canadian Pathologist out of mcgill that was more humble than this coder.

Did you read the bit where he said his projects "aren't impressive" and that he has "no intrinsic talent" and that he's just "reflecting on what has and hasn't worked for me"?

Or click on "emotional management" and read "In keeping with the theme of the rest of this series, I don't claim to be very good at this or to have any deep or generalizable insight"?

Or where he says: "I've never had to maintain a codebase for more than 2 years. I've never been responsible for running a long-lived service. Any thing that I report having worked well for me has to be considered in that limited context."?

Reply on news.ycombinator.com