If I Could Wave a Magic Wand
wilsoniumite.com
wilsoniumite.com
I will say it does help when slowly morphing code that it was all C code. C is very malleable, like clay. I've found code in some other languages I won't name to be less malleable, more crystalline, more necessary to smash to atoms to effect the desired change.
[1] https://lore.kernel.org/lkml/3DB30283.5CEEE032@digeo.com/
In other words, C is malleable like a metal, and oxidized metals typically aren't :-p
Years later I encountered the same trick inside Yocto: one of the few things done right which made life easier, in contrast to almost everything else inside that dumpster fire.
Start with the structure of what you really want it to be and then flesh it out. Soon discover that a lot of the existing code is OK, anywhere from a simple "for" loop to an entire module, because they work just as well in the new structure and don't have significant technical debt on their own. These can be pasted into the rewrite and possibly adjusted lightly.
There's rarely need for a truly clean sheet.
When I was working on Knot Resolver, I did a pretty successful rewrite of its I/O. The original one was a mess, but of course a lot of it did make sense in the grand scheme of things - it's only that the requirements had changed with time (mostly caused by the addition of DNS-over-TLS, DNS-over-HTTPS, and, in the future, DNS-over-QUIC), so various adaptations needed to be made to the original code. But, as the changes were largely incremental, it resulted in something that was pretty hard to reason about.
So I literally tore out the thing that I considered the most "wrong" about it, but sort of kept it at the side; then I started reimplementing it by mixing new code better suited for the current (and hopefully near future) requirements with the old copy-pasted code.
Spinning a bit off-topic, but perhaps sort of relevant to anyone planning to do something like this. The biggest gratification for that effort did not come immediately. Rather, the result was initially pretty underwhelming. Yes, the code seemed to be more readable, robust, and extensible, but since there was no immediate need to touch that area for some time, I started questioning whether it was actually worth it. But then, about a year and a half later, some new requirements arised, which proved that the effort was actually worthwhile. And we even had a direct comparison, because a security fix was needed for both the old, still-supported, version, and the new one, so it was nice to see the difference between what it took to fix the old one and the new one.
option 2: keep notes during onboarding. Revisit them after a few years working on code.
option 3: help new people onboard. Pay attention to where they struggle and if those parts could be improved.
I think it is because you are then able to assess the value of changing it - which is usually not very high. Also, when you are a couple of years in on a project, you probably have more important things to attend to.
I think it is great for new developers to attempt to fix these things when they come in: If they are able to, without breaking anything (and also convincing the rest of the team in a cheap fashion that they did not brea anything), then they know the code base and optimized it.
However, one should be aware that there is a high risk of them failing. Especially if the new developer is junior.
If you dislike a situation you're in and you try and fix it by switching to a new situation, you'll generally bring with you some of the problems that created that prior situation.
If instead, you bit by bit improve the situation until you feel at peace with it, you'll then either no longer want to move to a new situation, or if you do want to move, you'll no longer bring with you the problems of the prior situation.
Applies to job changes, relationships, projects, goals. And, from OP, applies to architecting software projects.
Personally I am a fan of switching to new situations in groups, as a way to push people out of their comfort zone and force them to account for things they may not have had the perspective to appreciate previously. People are generally resistant to change, but once they start to get caught up in it, it’s difficult to avoid growing from the experience.
This is almost universally true. We need morals/ideals, but they must be grounded in reality, in what is. If we just trudge along, we loose any vision for the future that could be better. If we just have idealism when we try to make things better, things usually get worse.
I'm not a constructivist (and definitely not a critical constructivist); I believe reality is knowable and consequential.
So have ideals; ground them in reality.
At work I often get tapped to work with folks who struggle with "Better Engineering" ideas (codebase improvements with an eye towards increased productivity). Usually it's just people being unable to come up with any improvements.
I always prompt them: 1. "Is this the best codebase you've ever worked in?" and 2. "If you were to rewrite this from scratch, would it look exactly like this?".
It's amusing how often those two questions trigger a light bulb moment. I of course follow up to ensure their ideas are actually good and grounded (no "let's convert the monolith to microservices") but it does wonders for inspiration.
Code that's not well compartmentalized and is full of complex dependency chains and flawed abstractions is hard to work in, and more importantly to the topic at hand: extremely hard to refactor well.
Once the abstractions are shuffled to their own "corners" of the codebase, and you've got well defined modules/services/microservices/foobars... you'll find refactoring to be far less of an investment. It also becomes far less attractive, as a well abstracted module is easy to ignore and forget about.
Of course, it's always best to make these things right the first time. Whenever I kick off a greenfield project, my first code-style objective is to make things easy to delete/remove.
Addendum: I find the worst spaghetti code comes from very dynamically typed languages. All the "easy" coding makes skipping interfaces/abstractions effortless, thus nothing's "doing just one thing well" and it snowballs from there. On the flip-side, when done right, it's quite a joy to write delete-able code in Python, and it makes prototyping and defining boundaries a breeze.
finding patterns that could be abstracted is fun like solving a tricky puzzle. or getting a chance to play with a new framework or language feature and just seeing what it's like.
i don't know how a manager is supposed to handle this situation, but as a dev once i realize what my true motivation is it becomes a lot easier. i can separate the 'coding for work' from the 'coding for fun' and just put my head down and do the boring work and look for the fun somewhere else.
Now, I think the 'no rewrite' argument has validity at certain levels. Systems will morph (because life/problems change) but the underlying functions may not need to. Composability and interfaces are wonderful tools to address that problem.
Of course, if you look at all those little deltas between the current and clean-slate states, some of them are much more expensive to live with or solve than others. This definition doesn’t help you decide which changes to make first, only what your North Star should be.
But fortunately, I write in-code comments documenting these, because I pay close attention to such details.
> 4. Your own code always feels better to read, because you wrote it. That doesn’t mean it’s actually better to read than someone else’s.
Having that mental context makes your own code easier to read. That's why those comments are so important: they share that context with the next reader. Well-commented code legitimately is better.
He tells me he'd be happy if it played at the same level but was faster (play more games) and also that if I want to take it to the chess club it has to respect time control. Supporting something like XBoard and UCI would also be a hassle in Python because it needs a comms thread that can interrupt a think thread.
I rewrote it in Java and the process was super-fast because I could cut-and-paste the data in the test suite also I had mastered the signs in the negamax algorithm (I screwed that up and it discovered this https://en.wikipedia.org/wiki/Fool%27s_mate !)
It's different from a lot of applications work because it's really a simple program and doesn't have the panopoly of features that you miss when you try something like
https://mobilesyrup.com/2024/09/24/sonos-employees-app-botch...
the hard part is that right now it is spending roughly equal time in evaluation and managing transposition tables. I think I can speed up eval about 20x which is going to make me code up some kind of specialized off-heap hashtable.
> The programmer builds from pure thought-stuff: concepts and very flexible representations thereof. Because the medium is tractable, we expect few difficulties in implementation; hence our pervasive optimism. Because our ideas are faulty, we have bugs; hence our optimism is unjustified.
Humans just have bad intuitions for this problem space, so you have to be consciously empirical; externalize decision factors, track outcomes, articulate hypotheses and be honest.
> I find that usually (good) programmers enter a new project with idealistic dreams of ripping out the walls; this could be done differently, that could be removed entirely, and so on. Then, later, the longer they stay the more those walls seem familiar, and the idea of changing everything becomes instead a distant memory.
Oh... analogies! In reality those walls will not disappear. Instead, the rooms that have been defined will become inhabited, and you'll notice that you just lack proper resources to remove walls. You'll start questioning whether you're in the construction business, interior designer (walls need decoration!) or just any other tenant for your landlord. You'll never be the needed tornado, so it's just best to hope to discover that the walls are actually made out of cardboard and some tropical storm is coming up after the past dry summer! :D
Fact seems, bad decisions can be done in a heartbeat, and it takes a lot of time and effort to iteratively leave it behind. I estimate that ratio to be somewhere in the 1/1'000'000 - 1/10'000 range and that's where I have some actual hope in LLMs to bring that number closer to 1!
My newest revelation to unwanted walls: If part of your code doesn't need to have a new release-(or life-)cycle, just don't put it in a new repository. The scaling advantages of microservices can be achieved with just a single repository.
All ideals are surrounded by a moat of tolerance.
How do you eat an elephant? One bite at a time.
Leave it better than you found it.
It's not crucial, but I just published a new tutorial[0], and, like all my tutorials, it taught me more than it would teach others.
I just like to actually implement stuff that I learn. It helps to "solidify" it, in my mind.
In the app we released, early last year, I worked on it for about four years, in all.
At about the two-year mark, it was almost "ship-ready," but then, I tested it against absurdly large datasets, and it fell down hard.
So I rewrote it from the ground up. Took a while, but we didn't have ship pressure. In the end, it worked beautifully. People really seem to like it.
But I would probably recommend against doing that, with most projects.
WFM, YMMV
[0] https://littlegreenviper.com/series/swiftui-charts-gestures/
Oh yeah, I agree with that. I wrote a program for work recently that probably should have been object-oriented. It would have been nice for it to be, because it's a bit of a mess now. But it works flawlessly and in truth it doesn't need any real feature upgrades now that it's done. So, I decided to keep it as it is.