Man spends entire career mastering crappy codebase
taylor.town
taylor.town
He's the guy you go to when you find some legacy code which you have no idea how the hell it works and end up getting a 2 hour history lesson into a decade of company politics and failed replatforming projects.
They're hard not to love.
The I fall asleep at my desk, periodically waking up shouting "STOP-A!" and hallucinating about being bitten by thick ethernet vampires.
Going to make myself a coffin...
When the Sun 4/330 I used to have needed a RAM upgrade someone flew up from London to Edinburgh with the RAM and installed it. Fortunately we were in the Grassmarket about 10 minutes from the Castle so he had plenty of things for him to do to keep him amused until his return flight.
Edit: I checked and the max the 4/330 could take was 96MB so I suspect the upgrade was an awesome 64MB. That machine cost more than most cars I have owned - even without allowing for inflation.
Do you know why I am giving you the two hour history?
First, I recognize that I need human interaction, but do not really like people.
Second, I am hoping that you will learn & understand instead of just memorize & regurgitate so in the future you can resolve it yourself instead of bothering me.
Now, go away; I had my week's fill of socializing.
/joking - or am I?
I'm at least going to be waiting for questions to be asked.
"Why was it done this way?"
"Circumstance combined with 'path of least resistance'."
Where does all this hope come from? I’m barely 15 years into my career and I’ve nearly given up…
I’m not trying to explain why you shouldn’t change it. I’m trying to help you understand what competing forces made it that way in the first place so you can decide if any of that’s still relevant.
But they are REALLY capable of detecting bad ideas.
There are really good or valid ideas that people try to implement in terrible ways.
things like: for our app on windows, we just ship LIBRARY.DLL with it. We should do the same with linux. (ldd shows 50 inter-dependent libraries)
Never mind that it's part of a decade old contract stipulating a quarter century support...
Greenfield is easy. :)
Because sometimes on the outside systems look confusingly written, but it could be a reflection of the scope of confusing business rules to begin with that they had to scope out
Seems better than being a 50 year old trying to wow a 28 year old interviewer with your skills on whatever hot new framework just came out.
Didn’t have to reboot the others, though.
That is still possible in the modern era.
They also hate each and every one of you.
I often search for a clearcut answer to a technical question and I'm met with a 2 hour history lesson into a decade of company politics and failed replatforming projects.
Yeah, thanks for telling me why John from accounting was a dick 10 years ago and you had to code this module in a certain way. I really don't care. I'm new to the codebase and I just want to know how it (the codebase) works.
I'm currently in this situation and a colleage never gives straight answers to anything. It's always some little rant about something and when it's done, I still haven't got my answer.
You should want to know _why_ it works that way too if you want to do any meaningful work with it. Context matters. I’ve seen many cases where the way something works seems dumb, only to learn later they had already tried the “smart” way but ran into some obscure problem which the “dumb” way solves.
Karl Hackerman shows up to each meeting complaining. He seems sour that he hasn’t been given the go ahead to rewrite the system in Rust. He knows a lot about bleeding edge frameworks and best practices and always seems annoyed that he can’t use them at work. He thinks it’s because everyone else has settled in to becoming wage slaves instead of striving to be the best programmers they can be. Everyone puts up with him because they know he’ll quit in six months anyway.
Karl is 47 now. Hasn’t held a job for more than a couple of years. Has mostly been freelancing lately and trying to sell books on tech and frameworks nobody cares about anymore.
I love this so much. I've totally been this guy :) Learning how not to be an insufferable doucheb*g was a surprisingly hard challenge for me
I've been writing a longer fiction piece about overzealous know-it-all engineers in the style of Confederacy of Dunces. Probably won't be done for a while, 'cause there's a lot more of my soul in it haha
Good thing I'm not in my 40s yet, still time to recover from my wayward path.
I felt this way earlier on, but quickly learned to get fulfillment in my day job purely from shipping and contemplate the mysteries of computer science largely on my own time. Sometimes they overlap, but good day jobs are often 80% boring in my experience.
The only reason to change such job I can imagine is to sit out your non-compete package. But I am not in this industry so maybe I’m missing something.
I also worked at an HFT C++ shop that used no STL. Every data structure has a weird name and API that you get used to. The whole architecture has its own idioms, and hasn't needed much upgrading in terms of language features.
Imagine you learn the grammar of English. You learn how to pluralise nouns, when to put an s at the end of a verb, how to to use commas, and so on. You then go to work at a place where they have a different word for coffee, drink, and order. Along with every other word. You end up learning where everything is and how to talk about it, and you're productive.
How do you apply for another job? Chances are you will be asked how to order a coffee and drink it.
Very real issue btw, I still have friends there.
Getting rid of weird data structures that now have STL analogs then is a productive exercise - existing devs get to learn more STL and new devs have more familiar ground. You get to delete code too.
> It never sat well with me. Colocated systems saw latencies no mom-and-pop investor could ever dream to achieve. That is one of the reasons why the second company I joined was a non-profit that built systems to help track human rights violations, enforce labor rights, monitor democratic elections etc. Third job was in the food industry and the last one, in housing / property management. So I'd like to think I've made sufficient amends.
Ah, nevermind. Turns out they were worshipping Memnon with C++ and renounced that cult with web tech! :p
I'm not sure if that's intentional humour or not.
If you walk into web dev with aggressive principles and a willingness to be controversial, you can get a lot of shit done and it will feel great. Watching others struggle under the various JS framework circuses has almost an opulent/aristocratic vibe in 2023.
At that level of abstraction it’s all just strings
Then I felt like I could spend my whole career there and decided there was a whole world out there in the web, began to learn PHP in 2004 until the mythical "blog in 5 minutes" Rails demo appeared. I switched jobs in 2007 and I have never looked back.
If you have access to key systems, you can definitely reverse-Conway to understand how an organization really works.
https://en.wikipedia.org/wiki/Conway's_law
Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.
— Melvin E. Conway
Hard to be productive when you can't trust the language, the tools, or other developers to be correct.
I'm very roughly estimating that you could probably have worked in the HFT job for around two-thirds the time you're going to have spent working on generic web tech, and still been better off.
EDIT: I'll reply to my replier here because /u/Dang rate limited me (again), though I must commend him for letting me have one more comment than usual before he pushes the button ;)
> I would say the ending -- the part where you are unemployed and unable to support yourself in your old age because you invested decades of your brainpower into a highly specialised, non-transferrable skill.
I would contend that such a person, unless very reckless with finances or extremely underpaid, should have enough money saved up to make what is mentioned there very unlikely. That said, I know some high-income earners struggle to not blow their entire bank account each month
Ouch. Should have stayed in legacy C++.
As God intended it!
Good for you I guess?
I can't fathom willingly entering the web mines if you've already sunk in the hours to master high perf native code, OSdev, dist systems etc. People usually enter web to avoid having to learn such things.
To discard that knowledge and instead compete with an endless stream of fresh grads, boot camp grinders and LLMs for less money? A madness if I've seen one.
Even if the decision he made was a bad one, you don't know anything about why it made sense for him at the time.
People aren't algorithms running in complete isolation and silence. They live in nuanced circumstances you can't possibly understand by his short history in a comment.
I hope you can find understanding and peace in the future.
If you want interesting, go to HPC large scale simulation.
same fate
> I'd like to think I've succeeded.
possibly not
I hate the thought of my life starting in retirement. Especially if I might be 67 by then, which is the legal retirement age, where I live.
When you say "legal retirement age", what do you mean? Is this to receive some sort of retirement benefit like Social Security in the US? Or is it something else?
- work 4 days a week only
- save 30–50% of your income and retire after 15 years
- cook your own meals to eat healthier and cheaper (bring lunch to work)
- buy used stuff from a couple years ago
- fix your own stuff and buy cheap “broken” things that are fixable
This way you might not need to die at work or wait 67 to retire.
More often than not though, its not so rosy. Arthur would be overweight, riddled with medical issues, solitary guy with unhealthy attachment to XYZ (anime, gaming etc).
But if he had kid(s) and raised them well, well as a parent too I do have tons of respect for him for that alone, rest are details when looking back.
My levels of ambition and relentlessness absolutely shifted after having kids, I used to be absolutely furious about being the best. Now I am very much "fuck you pay me" and my priority is having as much free time as possible for family and friends. I for one would be fine with Mr. Westbrook's career although thankfully I haven't had as crappy colleagues as him throughout my career.
There are people of all types who will want different things, no point in judging. As the great thinker Alicia Keys once said: 'you do you boo!'
I found out that just being better than yourself takes you a long long way.
I have never competed, but just did my thing. Then found out that I've already know more and ahead of many people I'm inspired by.
Yet, this was not my aim, and still is not. I'm just one of the so-called dark developers. I just do my thing, try to do it better every time, and don't care about hall of fame of anything casual or serious.
I like the mindset of trying to be just a little better than you were, every day. It feels... correctly scaled, somehow.
That resonates with me.
There is a saying where I'm from that says "most people are just kicking dirt" that basically means people are doing the bare minimum or don't care at all, and if you just do your thing and really care about your stuff, you're already ahead of the majority.
Being anxious all the time, I use that saying in the opposite way though: just relax and do your thing... you don't need to worry about not having a job, all those other people are barely doing anything and you can't be the best at everytime. Just do your thing are relax.
I usually hold vast disdain for the "self-help" genre of books, but after a recommendation from someone whose opinion I value greatly, I read Atomic Habits by James Clear. I've never found myself agreeing with and finding revelations from such seemingly mundane and obvious statements. Sometimes seeing a statement or opinion you take for granted can make you cognizant of that point.
I've since tried to adopt such "agile" methodologies in personal improvement. Clear says 1% improvement each day always sounds attainable, and such iterative improvements add up.
When I first had children, my pendulum swung way over to disinterest in seeking anything in my work beyond financial support for my family and time to spend with them (while still supporting and not breaking trust with my professional colleagues).
But my kids are growing and becoming more independent all the time, and I'm currently in a phase where my mental cycles and energy are on an upward trajectory over time, and now I'm back to being excited to use those cycles to do impactful work. But critically, I think, with significantly more wisdom about how to strike a healthy and happy balance for me and my family. I mean, I'm not naive, I don't think it will be easy to strike a good balance, I think it require be constant work and effort at it. But as I've emerged from the baby-brained haze, I have renewed clarity that neither focusing my energy entirely on family or entirely on work is the right path for me.
See I have the opposite take. There are so many damn facets of modern life/society, and new avenues/facets/domains coming out every year, that it is actually pretty easy to find some niche where something hasn't been done yet that is world-shattering for some specific community or niche.
World-shattering for everyone is pretty overrated.
For context, I recently wrote a tongue-in-cheek guide on how to coast in large companies.
[0] https://taylor.town/corporate-camouflage
That got me thinking about what happens when people, especially engineers, coast too much, or fail to plan out their careers.
I know plenty of smart people that are enchanted by comfort, then chain themselves to companies that take advantage of them.
If you've developed transferrable skills, this parable probably isn't for you :)
I personally think it's okay to be ambitious, and it's also okay to chill.
This is as much a tip for myself as for you: take a deep breath before reacting.
No there aren’t. For every Jonathan Blow, Salvatore Sanfilippo (redis) or Mike Pall (LuaJIT) there are probably thousands of run of the mill developers working at feature factories. You don’t build amazing software by being lucky. Your boss at GenericCo will never go out of their way to ask you to build that thing you’ve always been dreaming of making.
You choose to work on software you find interesting. Or, for any of 1000 totally valid reasons, you choose to be small.
If you think you’re all that, don’t blame your company for “not getting the opportunity”. Please. You already have all the tools you need to code. Make something cool!
You underestimate the odds of talent in a population of 8 billion humans.
> If you think you’re all that, don’t blame your company for “not getting the opportunity”. Please. You already have all the tools you need to code. Make something cool!
There are neurosurgery robots operating with my code running in the background for several years now. I am now working on ophthalmic surgery software for sub-retinal operations. I would think that is actually kind of pretty "cool!".
However, I still know full well that I am a cog in a big machine. I still know full well that my impact is not world shattering. And I still very much think "fuck you pay me!".
My self worth is pinned by family and friends. Work is fun but it's a side quest. If you think differently that is ok, and so am I.
This famous talk outlines it: https://www.cs.virginia.edu/~robins/YouAndYourResearch.html but theres a huge amount of factors that go into becoming one of those guys.
Its not just talent, or desire, or luck.
Its:
- Talent - obviously have to be really good to even do anything in the first place
- Vision - Need to have a goal of some sort to accomplish something
- Decision making - The goal needs to actually be something the world finds useful.
- Drive - Be able to take constant set backs and negative feedback and believe in yourself to an almost delusional level.
- Consistency - This means doing and thinking about your "thing" for a REALLY REALLY long time. Like weekends and holidays and ignore family and etc...
The talk above goes into it pretty in depth, but theres no mystery about it. You need ALL those things and no less. People who achieve at this level aren't accidents. They are simply not only talented in their fields, but also in areas of being able to consume volumes of information and handle failures. They're the ones getting up at 6AM every day working and thinking about their "problem" and going to bed dreaming about it, for years.You can be a 10x developer, but you also need to be 10x in several other areas to actually accomplish valuable things. Theres a grand canyon of difference between them and everyone else.
Equally, there are many, many talented people out there who never achieve recognition for what they do or were capable of.
People don't stop being talented/smart when they have kids/families/personal lives...but they will have less time and their perspectives do change.
It's a balance/spectrum where most of us end up somewhere in between two cliches: do what you love and you never work a day in your life on one end & work to live not live to work on the other.
Folks writing new code in the wonderfully free world without a ton of odd dependencies and corner cases ... IMO often aren't doing as much "work".
Working in an existing code base, on a team with multiple features going in, gathering requirements from stakeholders and negotiating deliverables on a timeline while refactoring is happening, libraries being upgraded and CVEs being fixed - navigating all of that while being a useful, compassionate team member - that’s priceless on a team.
There is "I'm OK with my pedestrian job but I still try to do my best and actually like the people there" and "my job sucks all joy out of my life and provides me with nothing of value except money".
I have kids, but I somehow skipped the "I want to be the best"-phase and went straight to "I'll make do with whatever you give me" so nothing really changed. I actually work a bit harder now so my kids can see work is good - also to pay for their fancy shit.
Even stupid jobs can be sources of relative joy if you let them. Don't put up with abuse of any kind, though.
Problem with the kids is that when I tell them about that, they feel they don't have enough passion to succeed. So you see, eventhough I was on the right side of success for a while, it scares the hell out of them...
10 years of being underpaid and yet being the most productive guy at work will do that to you.
Since having kids, most of these have shifted from a few months in time to a few years, but they are unambitious, poorly defined and unimportant so who cares if they take a little longer?
My main goal at work is not getting annoyed or frustrated and getting paid well for my time.
Pride, radical individualism, self-promotion. These are extremely common today, especially in the US (just compare an American resume with a European CV). Both the US and Europe have their problems, but this particular obsession tends to be more conspicuous in the US. Paradoxically, it doesn't bear the desired fruit because it misunderstands the nature of personal excellence.
This isn't a vote for collectivism or mediocrity. Only dead fish float downstream. It's a vote for the notion that the riches of society are the result of countless incremental contributions up and down the hierarchy of value for the sake of the common good. But liberalism embraces an inverted view of reality. All our relationships are viewed instrumentally and transactionally. We place desire above reason. We've become atomized, because nothing sensible unites us (perfect for those in power who view people as "human resources"). The "social contract" view of society is a disaster.
It's 2023 and today companies will quite happily lay you off the moment you aren't profitable to them.
This fictional person made it all the way to retirement and good for him. But the guy who followed this strategy for 15 years and now is job hunting because the company needed to cut 18% of its staff? He's got a huge gap in useful skills and decades left before he can retire.
That guy is in real trouble.
Arthur was happy.
To be fair: to modify blood dialysis systems is no joke business.
When I first read this poem I thought it was sad and about a person who wasted their life on mundane, average things. But now I think it’s a positive for a person who lived an average life on the outside but likely had a rich internal life that they didn’t share with the world and that we shouldn’t expect people to have metrics that we can externally measurable as to whether they were great or not.
I think it’s fantastic they this man’s passion is Soylent cat food hybrids for human consumption. He worked all those years and finally broke free for his passion.
The concept of life as meaningless, to be happy with the drudgery, or repetition, instead to be happy with the 'journey', to be in each moment.
Very common throughout literature/history.
I'd like to think we are somehow different, that modern man with technology is reaching some new level, to find some purpose. But now, not so sure, it's still just day-in-day-out, then you die.
https://en.wikipedia.org/wiki/Sisyphus https://en.wikipedia.org/wiki/The_Myth_of_Sisyphus
It seems obvious this guy was in fact happy, despite the last lines of the poem, or else he would have acted out at work, become an alcoholic, divorced, or whatever. That or a supreme stoic.
> Our researchers into Public Opinion are content
> That he held the proper opinions for the time of year;
> When there was peace, he was for peace: when there was war, he went.
I have a friend, let's call him AP, who is a programmer in a local java shop. He's been working in the same place for 12 years since he graduated. He kind of fits the description on this article, but he is one of the happiest people I know.
He gets paid above average wage, and his work is easy and not very demanding. He knows the codebase inside out, and gets along well with his colleagues. Outside work he goes out cycling or for a beer with his friends or travels the world with his wife.
Different people have different priorities, and some are not obsessed with excelling at work.
There are many fantastic engineers who work on otherwise boring, rotten codebases. No shame in that.
I agree with you :) here's some more context
Sticking to something, through good and bad times, on the other hand, that's not something many people can do, and while it is by no means sufficient in most cases it seems to be necessary for sustainable success.
Examples: Guido van Rossum[2], Linus Torvalds, Daniel Stenberg
[1] You wouldn't even have to put the "high risk" nature of it in the fine print, because it's readily understood.
[2] I could not find the source but I remember him saying in an interview when asked about Pythons's breakthrough, that it never had one but that it constantly and steadily grew.
If you're company is building a programming language or operating system you need to make sure that is what you love doing.
why dumpster diving? and those guys cannot be replaced by any number of junior devs - that is exactly the problem with them retiring. they know the legacy stuff nobody wants to touch but everything depends on.
so, from that angle this writing reflects very poorly on the author and his work experience.
having said that - the title could fit just about any lawyer. law is the crappiest codebase that you have to spend a lifetime learning. especially the german tax law - but it guarantees a very comfortable income.
also in defense of people like that man i'd even argue that mastering a legacy codebase is mentally and motivationally not that different form mastering a game like chess.
But, there are a lot of old people that are hungry, and work as Wal-Mart greeters.
If you didn't save/plan well enough, and 30 years of cool software development ends as being a Wal-Mart greeter, that can seem equivalent to dumpster diving.
I get that the post is satire but at some point in your career there are diminishing returns on learning new things. If you are somewhat specialized say in backend systems involving scaling, message queues, clustering, etc. it probably isn't a good time investment to learn whatever is new in the frontend world. And by learn I mean to become proficient professionally and get a job doing it, not just a "hello world" learning. Doing "hello world" learning on a technology is like learning how the yolk and rudders work in an airplane, you're not going to be flying professionally anytime soon.
I've spent 10 years as the lone dev at a small medical facility, writing database software and dealing with lab hardware etc.
I guess I've touched 100% of the codebase because I wrote it all, but I always wonder if my skillset is so niche that I'll always be at small companies.
Here's the deal though, work isn't everything and I'm at a job that recognises that.
I think devs imagine someone else out there is having a better time at their computer than they are. I definitely do.
He told his manager that the data access layer - consisting of 5k LOC batch scripts calling Microsoft Access - needed to be brought up to speed. Maybe they'd consider PostgreSQL this year. Containerization, he protested, isn't going to fix the problems inherent to the system's labyrinthian integrations with countless software vendors and proprietary components. He also reminded them that it's 2023 and Windows containers still don't support RDP.
They still needed a replacement for "Bob". Bob left the org back in 2008. Under duress, and impossibly tight sprint deadlines, Bob was asked to make surgical modifications to the WingdingsAbstractFactory.java class - to be deployed directly into production no-later-than-end-of-sprint. Bob complained that there were no conceivable DR plans, or QA engineers assigned to the project. After a ten hour deployment - and a highly blameful postmortem - Bob exited on record-short-notice.
Arthur was forced into an early retirement under the recommendation of his physician. It was said that his stomach issues were brought on by the stress at work. Only Soylent and tinned meat were within tolerable levels of discomfort.
Arthur will be remembered by his loved ones for his patience, steadfastness, and creativity.
I sometimes find myself pondering why I didn't cease working on it, especially given the compensation wasn't particularly impressive. It's crucial to recognize and act when you sense that a project is no longer benefiting you, and simply move on.
We get it. You're that guy. :-)
> Westbrook once touched legacy code without ending the company.
I'm cool with that. I keep doing work valuable enough to pay well for, and the rest is semantics.
30 years grinding it out on a very small esoteric system.
Hard to explain to people "no, really, my skills translate".
Paid well enough, but can't move, locked in.
But, the small esoteric system is being replaced, so I'm running out of runway and might not make it to retirement.
^^^This!
Arthur was lucky to retire at 58. He probably made fat bank which turned out to be golden handcuffs. If he ever got downsized, he had been f*d.
This is a big assumption, not everyone in this field is a high earner - plenty of people make okay-ish salaries without being anywhere near highly paid.
Many of the comments seem to assume that it was written by his colleagues and that the description of his career is condescending or snarky.
Personally, I find it really funny, and that it probably describes many of us, despite the fact that we think we are way too cool to ever be like the character described in the piece.
Most of today's new tech will become tomorrow's 'legacy'. Imagine how your awsum multi lambda hairball deployment will look in 30 years. (If it's even around in three decades.)
If companies want people to work harder, they'll have to give more compensation that scales better with performance.
He's doing well
Lots of significant inventions in computer science began as a side project only the creator cared for initially and had to borrow his own time to do it.
> Arthur had a knack for working hard without learning transferrable skills. He will be dearly missed
When I was at Google during the pandemic, I noticed more and more people saying they needed to learn open source tooling so they can have skills that transfer to the outside world.
That company is full of brilliant engineers who have mastered (part of) a great code base but have to start at the beginning with popular non-Google tools.
Or, put a better way, for all the codebases you have worked with, how many were crappy and how many were not?
I'm increasingly convinced that there does not exist a large codebase in actual use that isn't crappy in some way or another.
At least a 35yo codebase probably actually does something more useful than sell a narrative whose purpose is to enrich a few random jackasses.
Politics is a helpful lens. Either you compromise and muddy the purity of the result, or you take the high risk path of being uncompromising and greatly reduce the odds of achieving any result at all.
When facing this, many individuals optimize for lower-risk self-fulfillment over great ambition.
In any case it's just a job, and anyone harboring illusions otherwise will learn the truth soon enough.
I've been this guy, sorta.
I've been afraid to even try personal projects since then... that job broke me.
It's not rocket science - it's how we made webpages in 2001. Browsers are really good at that.