The Generational Divide in Software Developers
hackernoon.com
hackernoon.com
Responding to some of the points from the article:
* Insulating developers from interruptions was management’s Prime Directive.
Not even remotely, that concept didn't exist until the 2000s. The managers job was to micromanage those under them and people had to create ridiculous estimates and then put in crunch time to try to meet them. Management processes were based on managing manual labor. It was, in all ways, a horrible era.
* Projects were planned and designed. We had documents we were expected to read.
Sure, there were all kinds of docs that were incorrect, poorly thought out, and just plain wrong.
* Code is illegible.
The "golden days" were not golden. Code is far more legible now. "it was hard to write, it should be hard to read" was the predominant mantra of the 90s. Code these days is generally reasonably legible and clean.
The only aspects about the article that do ring true are some of the comments about testing - we did do testing and QA is great at least for complex domains.
This article is about 90% old man yells at cloud, and this coming from an old man ;)
Well, were you also from Microsoft? He seems to be talking about the way Microsoft did things "back in the day" - and it's hard to argue that they weren't very successful at it back then.
Indeed. Same here. I couldn't take the writer seriously. A lot of what he says is the result of the limitations of our computers, networks and software in the 80's and 90's. Of course I wrote Apple II code with manuals - we didn't have online docs back then - even in the mid-90's, I had MSDN installed on my computer for helping me write Windows software. I am not a better developer because I used books - I am an older developer because I started back when I had to use books (and manpages were optional and often not installed and it was nowhere near a given Unix would win the "OS wars").
The thing that turned the article from amusing to embarrassing was the tirade against writing tests and TDD overall. Writing tests makes me a better programmer. By starting to write tests as part of my design, my components get better - better usable and better testable and my software gets more reliable and better documented (because tests are documentation too).
I'll not say I'm an old man (53 is not that old) and while I can be bothered by some things I consider excessive framework churn (looking at you, JavaScript), I wouldn't say we are not a whole lot better off than we were then.
Writing bug free software without writing tests also makes you a better programmer. Practice helps, and practicing creating architectures that are easy to make correct without tests really makes you a better programmer. If you can't both do well with tests everywhere and do well without tests then you aren't a great software engineer. The reason is that first you create an architecture that is easy to make correct, then you add tests on top of that, and now you got something great, using tests as a crutch prevents you from ever reaching that stage.
Peopleware came out in 1987, and was read extensively at Microsoft in the 90s. You may be thinking of when Joel helped popularize it.
But let's say you work at a large plumbing company with 200 plumbers, and you have a tech department of 7 people and an admin staff of 25. The CEO is a plumber and what does he know about tech, the COO is working on coordinating trucks and insurance and personnel and all kinds of other things, and the CFO has his hands busy just keeping up with accounting. Nobody is going to mess with you, as long as YOU don't mess up. They don't even know how to help you, and they are too busy with their own sh-t. Except in the biggest picture view, of course. Then, all you have to hope for is that the manager is good. Usually they are, because they are busy. But I've worked at Fortune 500 companies in local offices or regions where I was the only person doing what I was doing, and absolutely no one bothered me, because I was there by myself. Sure, I talked to a lot of people in the company to see what they needed to have done, but after that, my time was my own. I can't imagine it would be any different now, in the same circumstances.
Which is ironic, because the "modern" incarnation of those things evolved directly out of an explicit reaction against them.
So what do you accomplish in the rest of that weeks? What does your org expect you to do? (Genuine question, I'm just curious).
I've been there. People schedule meetings for a living. It took 9 months to add a field once.
Enter bureaucracy, tickets, meetings, docs and more docs. More meetings. Their status report is more meetings.
Now as for IC developers only coding a single hour in a week, there's something very wrong indeed if you ask me.
I'm thinking "The Office" here. If you don't mind my asking what kinds of meetings did they invent for ICs to attend for 39 hours a week?
I think this is your problem, not modernity. In my 7 years working I've always stayed at smaller companies and I've never really had the problems you describe.
Now all of these are different roles. You have backend engineers and data engineers and data scientists and front-end engineers (who only deal with modern Javascript and its dozens of frameworks - even 10 years ago a front-end engineer was expected to know how a webserver and HTTP worked) and mobile engineers and AWS experts and Docker/Kubernetes experts to orchestrate it all. If you're actually responsible for the whole program, that's an architect or tech lead role and you don't actually write a whole lot of code. The jobs of each of the people who do write code largely consists of gluing together existing industry-standard frameworks. Algorithms and data structures are stuff you solve on a whiteboard to get the job, and then you never use them once you have the job.
It's the natural evolution of an industry, but it does make the job a lot less interesting. The observations the author makes are a natural outgrowth of this - when you have a lot of separate roles collaborating to build a program rather than a single person designing the whole thing, you need a lot more meetings and e-mails and testing.
We don’t talk too much about this.
1. How would a change be up in an hour if it required QA? 2. How is code review important but pair programming bad? 3. How is designing systems to be testable easily a bad idea? (I understand TDD is more than just this but it also implies this) 4. How does he reconcile his thesis with the explosion of Billion and Trillion dollar companies who embrace the practices he opposes. Clearly it can work. 5. Many of his practices are great for a Sr dev. But how do you level up Jrs? 6. The knock on Googling and SO seems off. Sorry for using some of the best tech ever made for dev productivity? Sorry we don’t solder our own chips? Sorry Googles top engineers built that mostly through pair programming? (Jeff and Sanjay). 7. Must be nice to just rage quit at the slightest annoyance. I am sure he is real nice to work with and discuss architecture with too. Also explains his disdain for the social aspect.
He also created Fit, which I had the pleasure of working with him on. It was a very different coding style than mine, very tight and flexible. I learned a lot.
Super easy. Just close cooperation with tester who's job is to test the change. You can with within an hour even with multiple back and forth.
In this type of 1 on 1 session with a customer, who was acting as a QA I managed to resolve more than 10 issues on the fly within an hour. We had multiple sessions like that and they were very productive.
Some issues couldn't be fixed "while you wait" of course and had to be postponed and checked and fine tuned on next session.
Those bugs had limited impact so the QA could manually test all the reasonable places of impact after my fix, while I was fixing the next one. It was really great feeling of interaction and collaboration. I felt like a proper wizard and I knew that after this sessions all the things we fixed are good.
This is completely incomparable to TDD, where you just sit in your own muck and shovel it back and forth with very little interaction with outside reality ending up with ton of code that does nothing because it just makes tests that will never fail or they fail all at once because they will fail only if something very major breaks.
I worked at Microsoft for about 12 years and when I joined I was an SDET (sort of a dev who does testing). I honestly found it strange that there was a separate role to test code that was written. It just seemed so wasteful. There were strict "QA testers" as well, but they worked with the finished product as opposed to writing code that tested other code.
When I finally switched to the dev role at the company, I would routinely see other devs writing code and throwing it over the fence to be tested. Testers would find an issue and then throw it back. Many of these bugs were such obvious things that could go wrong to me that it was clear the developer didn't take 5 minutes to try scenarios other than the happy path.
I do think you can go too far with TDD and unit tests and that often they are more trouble than they're worth to set up and do prevent you from moving more quickly. Often, you don't know your problem space well enough to start planning your tests. However, it's a very old school attitude to think that you can just rely on QA to do testing for you. It may have worked in the past, but only products took longer to release and there was a smaller audience for what you did.
Being a great tester requires very different set of skills than being great developer.
At a company that didn’t have remote workers, we let one QA person work remotely because she was that good at her job. She was finding more bugs than the next two people combined, and we had three people we considered to be “good”.
That is why it's so valuable to have dedicated QA testers.
I don't know how many times I've missed some simple thing, even in HTML that someone else spots in a few seconds.
I also agree with the point about "Blind Spot" from the author. It is just another perspective. Software is used by multiple users and "it works for me" is not encouraged unless we are building for one user. So, we need to get feedback from others and improve to address more kinds of users.
It's pure arrogance that developers can do anything, including testing.
And automated tests only test things that developer thought about.
If anything developer of given piece of code should be expressly forbidden from delivering automated test for his code and other developer should be tasked with this with guiding goal of making the test fail on the initial implementation from the first developer.
Forbidding is nonsense. A. test is also form of documentation. Nothing stops you to add more tests on top of those by different person.
It doesn’t mean no QA is done at the end. It’s just less overhead to also do it along the way.
This guy worked at Microsoft in 90's - 00's, and claims that it's the lack of concentration these days that leads to buggy code. I've used a lot of MS stuff in the last 30 years and there were many bugs and instabilities.
One of the most brilliant things I use these days is our CI/CD pipelines, which has enabled many individuals to collaborate, and deploy (mostly) reliable code to production frequently, daily if not hourly. Tests are critical in enabling this.
Perhaps the poster's views make more sense for waterfall-ish, once every 6 monthly or yearly releases.
And then you look at today when hardware is super reliable, people have forgotten that running out of ram can lead to bugs etc, yet we still have plenty of strange bugs in software that never should have had any bugs in the first place. If people today learned systems and wrote things with the same care they did back then you wouldn't have all of those bugs. But instead shipping bugs is fine etc, we took all the advancements and used it to push features faster rather than make things reliable.
In those companies I've worked where we released in some waterfall-ish manner and/or QA was important, we have both.
I have a brilliant coworker, who enjoys the new way, and we disagree a lot, and I think the article summarizes our disagreement very well.
Ultimately, I think the divide comes down to your optimization horizon. In the old days, software was made to last longer, and there was more time to plan and design it properly, because the horizon was longer. Today, the horizon has been shortened, and more experimentation is done, and less forward thinking. And so the methods are different, and most of these differences (as described in the article) accounts to the shortening of the optimization horizon.
I do prefer longer horizon, because I feel that if you have series of short horizons, you're not optimizing as well as you could, and spend too much time rebuilding stuff in an incoherent architecture. But the business seems to prefer the short horizon, for various reasons (one might be simply competition, the red queen effect of sorts). I think from an engineering perspective, this is a mistake (there is a reason why we don't build houses in "agile" way), but I came to recognize that neither side is actually "right".
But I am going to appeal, if you are trying to build software that lasts, think about your time horizon, maybe you will find that you can actually use the past approach better, if your horizon is longer.
I think that's the wrong way of looking at it (or phrasing it). I'm sure people that pick vegetables all day don't like the heat, or the dirt, or the bugs, or the fact that vegetables grow so low to the ground, but those things are immutable reality that can't be changed, so they have to adjust to them. Nothing about modern "Agile" software development is immutable so the better question is "does this produce better software?" It should be abundantly clear at this point that it does not - that although there were deficiencies in the software of the 20th century, those deficiencies pale in comparison to those of the modern day, in spite of the fact that our computers are orders of magnitude better. It doesn't matter whether we like the way things are done, what matters is that it's exponentially inferior.
Honestly, I am not so sure. Yes, I feel the same as you, but I might be biased, there are so many people who swear by the new approach. I don't think anybody has a definitive proof one way or the other.
"I've never done waterfall" or "we specified everything (on paper! millennials don't read amirite) before we built it", which is it?
There's a whole lot of generalizing from personal experience to "universal truths" that simply aren't universal. or even truths? For instance, there are certainly some teams that get value out of code reviews.
Were you “there” though? I wasn’t so genuinely curious if his recollection reflects reality.
EDIT: they responded to another parent comment I created and were definitely working at Microsoft during the period.
Yes, that's correct, I've never done waterfall either. By definition [1] "The waterfall model has, at least, five to seven phases that follow in strict linear order, where a phase can’t begin until the previous phase has been completed." (emphasis mine) - we never did that, and I never heard of anybody following that model. All these steps has to overlap - it is not possible to write good requirements without having at least draft design etc - and they surely did.
Firstly, where do you stand on developers having input into the design process? From my experience, developer input in specification has led to better fitting solutions. Personally, I wouldn't like to be in a situation where I am instructed what to do, that sounds extremely tedious.
If you agree that developers should have some contribution to the design process, then two points in this post are in conflict. Either the developer must be interrupted to participate in synchronous design decisions, or participate asynchronously on a much-maligned platform like Jira (better alternatives available). Is there another way?
Elevate “developers” to “Engineers” and treat the situation like building a bridge or a tunnel or an aircraft.
Of course there will be some small changes along the way, but that doesn’t mean we should hold twice daily ceremonial meetings.
Coding on quicksand will never work out.
I was the programming/tech half of a small company back in the MS-DOS days. The first cycle was 3 months (delivery of program/hardware as specified). When it turned out to meet specifications, but was impractical in real use, we agreed to make changes. The prototype took about a month, and the customer liked it.
We then switched to a really interesting form of agile. Every day I drove out to the Will County Generating station. The customer (Russ) would bring in a random generating plant employee, and tell him "Here's a computer, I want you to do X,Y and Z... I know this isn't your job, and you won't be judged if there are problems... This is Mike... anything that goes wrong is HIS fault"
Russ was an amazing teacher, in the end. The first thing I learned is that "Press F1 for Help" should always be visible on the screen.
We quickly settled into a pattern. We'd build a list of bugs/design changes, and I'd fix all the problems in our list, and then we'd test again... after a year it was judged suitable for wide distribution. I loved that job, I supported that program for about 5 years, driving all over the place, meeting interesting people and solving their problems.
I had a very similar experience building a scheduling app for a UK freight company. They wanted a domain-heavy platform where they could manage their fleet of locomotives and wagons, with a lot of computer assistance.
The product owner was a gem. Once he warmed up to the process, we'd get him in a room and he'd talk and talk and talk about everything he found difficult. I mainly worked on the project alone, and bit by bit we addressed his issues. The issues grew more and more specific, until about 3 months post-release, where he came in and had nothing left to add. There wasn't even a wish list left. It was done.
This was where I came to agree with the agile 'small batch size' approach. Mistakes come cheap and fast. Most mistakes are usually communication issues, rather than technical. That's not to say that requirements were changing while writing code - it's just that we had good feedback loops, and a short phase had us fixing the real problems faster.
- Teams do CI/CD but most teams don't do Continuous Integration (sync and merge daily or more). And they hardly understand why CI is important since that is how they have work for ever.
- "Do The Simplest Thing That Could Possibly Work" has been forgotten. People add a lot of complexity that is not really adding a lot of value (See YAGNI)
- Agile were supposed to re-capture the power and give it to the developers. That didn't happen. There is a lot of emphasis on the process but little on the principles.
> I and everyone I have ever talked to about testing has had the same experience; we write a new feature, test the hell out of it, can’t find anything wrong; ask a coworker to take a crack at it, and he finds a major bug in under a minute. The password is blind spots, and we all have them. And if we didn’t think of some case in development, we are not going to think of it in testing. That’s why we had separate QA departments.
I wouldn't go as far as the author saying TTD is useless as it's genuinely useful against most obvious bugs, but it's only a a part of the equation. But it's not like all companies removed their QA departments either - many pieces of software are regularly tested by independent teams with good results. If someone thinks TTD will eliminate all bugs, well, they're not thinking clearly.
I am skeptical of TDD, so I have a genuine question. Let's say TDD is able to find the "most obvious" bugs, like a typo. My question is, why are these bugs worth creating a test for them, compared to either manual testing (running the program and looking at the output) or running a portion of the integrated test suite on the code to find them? If the bug is simple and crippling, then the program is likely to fail anyway.
To me there is a tradeoff. With TDD, you write some unit test code, and it finds a trivial bug. But is every trivial bug found worth maintaining the unit test code going forward? I mean it's not like this bug is gonna appear again all of sudden in your code, unless you change it, but then in all likelihood you need to change your unit test as well.
I guess I don't really see TDD as a good tool, because all the situations that you can run into are already covered by other, more specialized tools. If you want to catch edge cases, asserts are IMHO superior tool to tests. If you want to verify your program quickly, running it manually and looking at the output is superior. If you want to verify your program properly, running a comprehensive (and integrated) test suite is superior (the best if you can make it property-based).
There are situations where unit tests are genuinely useful, like writing a library. But in those cases, unit tests seem to always either compare with some other implementation (for example, if I am testing a sin(x) function, my other implementation is the calculator), or at least compare with well-defined specification that comes outside the code. But I feel like majority of tests written for TDD are not really testing the code against anything (simply because another implementation or specification is not available), they just test the intent of the code itself as is written, which might be useful if you wanted to write "bar" but wrote "baz" by accident, but other than this type of bugs, what is their long-term contribution, that could justify the maintenance cost?
TDD tests would ideally be used for hunting business value, and focus on overall desired product behaviour and qualities.
I don't think it really helps software development, if instead of "here's the description of what we need to do" the SW developer gets "here's a few examples of what we need to do" (albeit if the examples can be automatically verified).
If that's NOT your intent, what's the role of specification in the TDD, and how it's interacting with the implementation and the tests?
The adherents really want tests to be the specification, as a way to be measurable / testable. Specs may be converted to tests.
But most of it is theory. It all depends, as usual, on the dev(s) and circumstances.
That's pretty much it. The value is that tests can be automatically executed and verified. A specification document can't.
Writing a spec is still a step in the process - a bootstrapping step. The spec is a "write one to throw away" placeholder until you get far enough that the spec can be replaced with executable tests.
My take on specification is that you encode it to be executable as your program, during development. Ideally, the program should encode the specs in the most straightforward way possible. To "specialize" the spec into tests first and then "generalize" it again from tests into a working program seems like unnecessary hassle, and frankly prone to information loss.
It's interesting, because as I state in another comment, I have a big disagreement with a guy who loves TDD. I guess I am a theorist (I derive my feeling of correctness from simplicity of the code) and he is an empiricist (he derives his feeling of correctness from many small tests), and it somehow defines our approach.
The barrier means QA doesn't have the same model, and often lacks info that you have. That's valuable in some aspects, but it's not double-entry. (Double entry in accounting means that you always record two transactions that need to match each other, not that there's independent verification - that'd be auditing)
I believe you need both double-entry and independent verification. (Of course, that depends on scope & desired quality. Your 100-line bash script that you only run to print fortunes for your friends probably needs less rigor than products that are used by billions, or that have life-or-death impact)
You're right that it's not specifically TDD that enables this - but you do want automated unit tests written by the developer, whatever methodology you use. (Or at least I want them :)
Unit tests still help of course with static typed languages but you really don’t need to be afraid that refactoring can kill your entire code base as you have formal proof that types and names agree as part of your IDE and build process by default.
The obvious bugs should already be accounted for if you have decent param checking in most functions. Is the string empty? Is the array nil? Did the init succeed?
Inline unit tests anyone? :-)
I’m honestly curious as I’m too young to know myself and I have no way of knowing whether the article was written with rose tinted glasses on.
But wow if it’s accurate the point about meetings and concentration sounds amazing, would love to go back to that.
He's not making things up, but I don't think his takeaways are worth, well, taking any further.
I actually snorted at the essay's idea that communication and concentration can't mix. I'm not sure what he thinks email and detailed specs are if not communication. I think the key is that they're batch-mode communication.
I can see value in several hours a day of concentration unbroken by communication, but insisting on a full week straight of unbroken concentration is impractical and possibly counterproductive.
Of course, that may be what drew me to coding then. Perhaps the new culture attracts a new type of coder that likes process much more.
1. Open offices are stupid. For proof just see how many people love working from home where they have far fewer unwanted distractions. Yes there are other reasons to like remote work too, but I bet lack of distracting surroundings is up there.
2. Not having dedicated QA is mega stupid for something like Windows. You can clearly see that in Windows 10.
Other than that, yes I am a bit more accepting of some of the newer stuff, but I get where he's coming from. y'all need to get off my lawn.
But it's truly bizarre that they're not, and companies think that is OK to put people in an environment that is the completely opposite of the environment it should be.
In 1990, real software companies could get away with shipping something major less than annually. Companies paced development to events like the annual Comdex. No company was expected to ship production software weekly, let alone daily. Competitive threats arose more slowly because there was less capital. The US software industry could be reasonably sure a Chinese firm wasn't going to suddenly arrive and become a material strategic concern in a few quarters. Seriously, read books describing 1990s era software dev and see how slowly things are paced before the Internet really hit. Look at the time scales involved. Even the rise of mighty Microsoft, considered the poster child of rapid high-tech growth circa 1992, pales in comparison to the Internet giants.
The world sped up. I hate the meeting culture we have now, but it's important to understand that it is a response to the pace of change rapidly increasing over the last 24 hours.
The other thing that really happened is the consumerization of software. In 1990, a much smaller slice of people used any computer in a given day. Frequently, those people could be expected to have undergone specific training for your product. Today, the number approaches 100% of adults globally. This means a higher level of fit, finish, and robustness are required now versus in 1990. Expectations are far higher.
Third, and I can't believe this needs to be said, but Microsoft was known for especially buggy software during the glory days highlighted in the article. Their OS releases were frequently late. They had to abort/restart the Vista project. Aside from the aesthetic factors of crashy software, it is somewhat obviously bad to have a multi-billion dollar company be unable to predict when (if?) its flagship products will ship, and in what form. Obviously, something had to change.
There's a lot to learn from the old days, but if one doesn't understand why things changed, one is likely to take the wrong lessons from history.
Yes. Our customers were, for the most part, nerds like us in the 90's. We knew what features they wanted because we wanted them.
Ask me about the time we had an informal meeting to decide on the premeeting strategy where we'd define the strategy for the actual meeting which would then result in a presentation to management so a decision could be made :)
It's always been a question of the culture you're in, and what they're trying to do. It still is. More collaborators => more meetings. Less ability to tolerate bugs => more formal development processes.
There's only a limited set of things where you can bang away in your dev cave for a few months and emerge with something people actually want. (It's absolutely more fun than the more formal/meeting-heavy approaches, but you're limited in scope and quality)
(With the caveat that open offices are and always have been an extremely stupid idea, and those were indeed less common)
We had "team (tech) leads" in the old days that took the place of marketing more or less — setting the general course for the software as a whole, the architecture as a whole.
But each engineer was given their own sandbox, their own piece of the framework/app to own. "Go implement an image cache."
Image cache engineer got to style their code however they pleased. They could tear it up and rewrite it when there was excessive technical debt. They knew it inside and out since they wrote it.
Often if it was code they inherited they would end up rewriting it eventually regardless — maybe piecemeal.
I don't remember every having code reviews. Sanity check? Sure, when there was some tricky bit of hacking required.
Yes, QA. Unit tests seem to make management happy these days. "I want to see 85% coverage!" as though that magically means we have less bugs.
I worked at Microsoft and do remember when everyone had an office and the goal of managers was to keep you from having too many meetings. Nowadays, I also have probably 1-2 hours of recurring daily meetings. Absolutely everything needs to be tracked (daily standup, sprints, 1:1 with manager, skip-level meetings, team meetings, all hands, peer feedback), so you end up with meetings just to make sure everyone is on track. It's ridiculous.
I think it's all related to the fact that the pace of software development was just so much slower back then. I remember you could really bullshit a lot more 10-20 years ago. You could hang out with a coworker for an hour or play games with other coworkers and it was acceptable use of time. Nowadays, with daily standup, I feel guilty for wasting too much time because I'd have to give an update on where I am with work. Before, you might sync once a week, at most, so you had some breathing room. The industry was just more "fun" to be in.
Product releases were on yearly, or multi-year, schedules as well. Now everything is updated every week. I remember even ten years ago, we were releasing our software every 2 or so years, then they worked towards once a year, and then 6 month updates, and then monthly, and so on.
But this gem stands out:
"I would fix it, he’d verify, and we’d keep no records."
Yeah, you don't get to work on my team if you're going to sweep bugs under the rug.
Companies varied tremendously. I don’t share many of his opinions and experiences, and I’ve been coding his entire career.
I LOVE pair programming, for example, on the rare event that I can convince someone to do it. And “waterfall” did happen, but was mostly Fortune 500 and government contracts, and those of us not in that situation laughed at it. And I see benefits of test driven development, even if I write most of my tests after the fact.
The number of meetings, in my experience, was more a function of the size of the company and the team, than the era. I suspect some of the change he blames on millennials is just Microsoft getting more bureaucratic and bloated over the years.
One thing that I haven’t seen anyone commenting on is the huge shift in how software is delivered. If you take a year or two to plan for a physical release of a product, there is a lot more pressure to get things perfect the first time. If you can patch something today and go live, it’s not the same as have to send physical updates to a million customers. QA was definitely important, and, I would say, more important than today. Developers as testers is great, but by no means enough. I think testing has been shifted to users somewhat.
I do think that there was more of an understanding that uninterrupted time was very important. IBM did a lot of studies on what produced productive programmers and uninterrupted time was a huge factor. But software development has changed hugely over the years. I’ve been building a project that is similar to something I built only 15 years ago, and it’s amazing to me how much easier it is. I’m spending way more time gluing components together, with the help of stack overflow, and less time on hard problems. This may have changed what the optimal strategy is. I certainly try hard to be a good programmer in today’s environment, and not waste time complaining that things were better when we carved software into stone tablets.
So, I’m not going to say he’s wrong, but a lot of that was not my experience.
I do though fully agree with many of the points in the article, and desperately wished we could actually change the corporate culture around SW development.
However I've come to the conclusion, after years of trying, that it simply can't be done. At least not in companies that have reached a certain size.
The 'agile' managerial mantra, along with all of the other bad practices that usually accompany it at cargo cult workplaces (managers that constantly work on 'optimizing' procedures and creating 'effective' teams, too many too long meetings, excessive documentation, lack of space to think and ponder, lack of private silent spaces, etc), is simply too entranched in the manager layer and company culture at these places to ever change.
The only times I've had the option of silent uninterupted deep work (i.e flow) has been when working at startups that were started and run by other SWD's who knew what worked and what doesn't.
And regarding this point:
"Instead of distributing projects between “teams” (sorry, that word makes me think of sports, we were “groups” back then)..."
I believe this is done exactly to make it feel we are like a sports team competing with other teams for a price. It makes it much easier to sell layoffs to the rest of the team when they happen ("we had to let Steffany go, she just wasn't an 'effective' 'team' player")
Edit: I will add that there are many good points in the agile manifesto that I belive would make sense if done correctly. The market place of 'agile' consultance has simply twisted it into a unrecognizable mess and so the corporate culture trying to implement it falls horribly short (and in the wrong direction) of doing so correctly
I wonder how many devs can code for more than a hr without being connected to the internet. I had the pleasure to talk to some industry veterans who have been coding for more than 4 decades. It amazes me to see how they could code without constantly "needing" to be connected online and actually build stuff just by reading through reference docs and man pages.
Atleast in my line or work (web dev), lately software dev has become more of team sport where one constantly glues things together and searches online in stackoverflow, github etc to look for answers or raise bugs with downstream projects. Dev itself seems to have become more Operations ish.
If projects were documented — like, actually documented, not just a cursory README — we would all be able to program without the internet. Honestly, it’s much easier than the way things are now.
It's too bad it was posted on HN over the weekend (and looks like it might've fallen off the front page pretty quickly), but maybe it will get a second chance in HN primetime.
It almost makes me wonder if comment timestamps are stored relative to the parent post...
Tested subsystems are substantially more valuable than untested ones. In my own IC work, related to my specific area of domain knowledge, I do not submit untested designs. Everything I design comes with a test process, which usually involves code that I write myself. My designs rarely come back to me. For this reason, I support TDD in principle, though I haven't learned enough about it to be great at it.
"Don't interrupt the programmers" is a problem because they end up becoming isolated from the rest of the team, and have to work twice as hard to catch up when the hardware design gets thrown over the wall for them to deal with. The people who can handle interruptions become by default the ones who participate in the critical technical decisions.
Also, what the article doesn't take into account is that the hardware, SW tooling (and its cost) and, most importantly, SW distribution methods have changed radically.
Lack of uninterrupted focus is the only really valid point, in my opinion.
I imagine working from home could be done in that setting.
I will grant the possibility that I have bad habits when it comes to work and I get easily distracted when I'm programming by myself. Having someone there effectively mitigates this and the lifestyle choices that negatively impact my ability to focus for long periods of times. So perhaps there's an untapped flow state that is even better than pair programming.
While I figure this out for myself, I still can't deny the fact that the last few times I've truly been in the zone have been with someone looking at/hearing me code.
1. Verification is automated - From my perspective, tests predominantly serve to document AND verify the behavior of code. You don't have to follow TDD to buy into the idea that tests serve to verify behavior of the code that was written. Some applies to formal verification of software.
2. Documentation stays in sync with tests - I'd imagine that any snippet of code is read 2-10x more than it ever changes and tests help with that reading process. Writing tests means you're optimizing for the thing that happens most often AND you are somewhat guaranteed that your testing "documentation" stays up-to-date because as software evolves the tests will need to evolve as well there is an immediate penalty for not updating the tests; there is no corresponding immediate penalty for forgetting to update a specification document. The older a piece of software is the harder it will become to understand: the original developers stop working on it, the documentation becomes stale, context and initial reasoning for certain decisions is lost. In such an environment it's, in my experience, generally easier to make changes to old code bases with lots of testing because generally tests will tell you if there is something you don't understand with nearly instantaneous feedback: write some code, run the tests and it either works or it doesn't.
Similarly, I don't get the authors argument that code is more illegible than it has been in the past. More modern languages make this as easy as possible to get correct. Take something like Flask [1] written in Python and generally considered to be well-written and compare that to the C code in SQLite [2] which is generally considered to be well-written C and tell me which is easier to understand without any prior knowledge.
[1] https://github.com/pallets/flask [2] https://github.com/sqlite/sqlite
What went wrong with agility is that it turned into Agile, a concept sold by consultants to be co-opted by management. We should absolutely stop coining the phrase "team" though. Everything about it and the rituals are only for enforcing control and disrupting creative work. The "sports" metaphors are only avenues of manipulation and abuse. Sadly it works best on the youngest generation.
The old way where a few devs could hold entire companies hostage is over forever though. Nothing wrong with pairing and teaching others. But it is work, work both the new way and the old way is very adverse to do.
Other things mentioned I just don't think are actually true about modern software development, although possibly it depends on your company. I've worked mostly in FAANG companies and we absolutely spend a ton of time on design. And I think this is where collaboration, team cohesion, etc. is so important. The design phase is where you need that communication, then you ideally separate and focus on your own work solo. Then maybe get some review etc. (though like he mentioned, I agree code reviews themselves are relatively pointless, I haven't done one since working at a smaller company). But during the design and somewhat ongoing it can be extremely helpful to know what partner teams and engineers are working on (though daily updates absolutely are not needed).
My biggest takeaway from the article is that he really, really, hates writing unit tests. I see pros and cons for both approaches honestly. As a software developer I would love someone else (dedicated QA) to handle a lot of this for me. I don't know if that's necessarily actually better from a company perspective though (not saying it's not either). It definitely makes scaling tougher for very large companies. Who's QA on a 3 person team (or "group") for example, does every team need dedicated QA? When is it worth having? Obviously that's something that can potentially be solved, but it's not necessarily easier or worth it. My experience though is that at best it's extremely exaggerated to state that unit tests are the main focus these days. In total I maybe spend two weeks a year writing tests, if that.
As for documentation and learning on the job my experience also doesn't stack up, and this is maybe the part I feel like the article comes closest to "old man yells at cloud". You are absolutely expected to constantly be learning and improving on the job and you better have read all relevant documentation. The first thing that happens on any team I join is I'm given pages and pages of documentation to go through. I spend most of my early time on teams literally studying it and taking notes. I think a lot of the "back in my day" quotes in regards to these topics are some of the most out of touch in this article. "We didn't have stack overflow so we had to..." So what? Stack Overflow exists now, people adjust. It can also be a learning tool in and of itself.
Overall I think some of the major points may be true (though as mentioned I don't necessarily think younger engineers would actually disagree with those points). I've never worked at Microsoft specifically so can't say what it's like there, maybe a lot of this is more accurate. I don't think I read very much that I think actually shows a divide between software developers of each generation though, other than speculation that younger developers can't concentrate.
I have not worked in the field for a long time, but everything seems the same back then as it does now, as you represent it happening.
Back the the day, of course we spent a boatload more time in the design phase, lots of meetings to see what needs to be done. That's just common sense. How can you program if you don't know what to program? But then, as you get into the project, you create an excel spreadsheet with all the questions that you have, and then get them answered. But I would HATE to have to go to a scheduled meeting every week to discuss it. I prefer to just go over and ask the people, when it was convenient for me and them.
However, I had always worked in smaller companies, where it was easy to just walk over and talk to the person, no matter what level they were, from receptionist to CEO. But I recognize that in a huge company like Google, things have to be different. But personally, I would never elect to work in companies like that. It's not my skill set. I'd commit suicide in a company like that. I'd rather work in programming in a sewer company or a retail store company with 20 stores, where they had a computer department of 5 or 10 people. That's just me. I hate being a cog in a machine. I like variety and talking to all different levels and different departments, when I chose to do so. Nobody in a small or medium sized business is going to have mandatory meetings 20 hours a week. That's just not reality. Just no way in hell, back then or now. As I said, I have not worked in the industry for a long time, but I think you can generalize this, that working for a small or medium sized company is going to be as he said. But even back in the day, if you worked in IBM or some huge company with 50 or 100 programmers, even then, it was like the author said he hated - much more strict and regulated, even back then. I never worked in them, but I was in the industry and talked to those that did work in them, and read industry journals. But even at Microsoft, there was a f-ck of a lot more rigidity. Maybe not if you were a superstar programmer, but if you were low on the food chain, you were going to be strictly regulated. Maybe Microsoft not as much as IBM, but still. Not like working in a 3-15 person department at a small company, where you pretty much have full autonomy, and your time is your own.
So, things really are not too different, actually.
I see that you worked about FAANG, but even now, would you think working at a small company with 3-15 tech people, would be more like the author said? Just by common sense? I'm sure you agree with me.
The point about focus is very true. The socialization of development via online tools means people think sticking their nose in other people's code is a virtue instead of a bother. If you're not going to contribute, why are you interrupting me?
The only thing worse than working in an open office environment is having to sit next to someone who stopped showering and regularly smokes.
Smokers/Alcoholics/drug users/the unwashed are the worst, followed closely by people that wear heavy musky scents.
Just one example: "Thirty Years Ago...Insulating developers from interruptions was management’s Prime Directive."
From my own experience, when I started at IBM in the mid 1980s, we had private offices just for this very reason. Me and my colleagues also recognized, though, that we were on the tail-end of those good ol' days. By the early 1990s "bullpens" were becoming the rage, and the rest is history.
Another, "QA was other people": it was other people, and not the can't-make-it, under-performers, either. Testing was a discipline in and of itself, with very experienced testers. They were often older programmers in their last 5-10 years which knew what kinds of corners and edges to think of. That, too, was a just a fond memory by the early '90s.
But we also pair-program — when it makes sense — and we practice TDD.
Perhaps the author has not observed good TDD. I find TDD enormously helpful; TDD helps me get into flow, and maintain it.
I think there’s as much wrong with this article as there is right, and can’t really recommend it.
A lot of what turns me off about TDD is the dogmatism. If I'm coding a bug fix, I will often start by writing a unit test that reproduces the root cause of the bug because I don't want a regression, and I want to be sure that my unit test actually catches the bug before I fix it. If I'm writing a pure function and I want unit test coverage on it, I might as well write the unit tests first. But it's not necessarily useful to write all of your unit tests first.
In doing "solo, meditative work", the hardest skill you have to develop is keeping your mind trained on the direct thing you're trying to accomplish, and not letting other parts of the problem intrude on your though process. If you've got any drop of ADD/ADHD (which by the "any drop" definition basically includes everyone), your mind will constantly get pulled away from the "task at hand" and start thinking about other parts of the problem. You'll be working on subsystem A (i.e. the task at hand), and you'll start thinking about how it interacts with subsystem B, and start thinking "oh gosh, I didn't realize this will also impose some constraints on subsystem C".
One of the key reasons this is a mind trap is because these side-things are genuinely useful; you'll run into caveats as your mind starts exploring "the interaction of subsystem A and B", and start to realize B's actually going to impose some hard constraints on A. The naive thing to do is to assume "well, there's no point in designing A because of what I just discovered - I better figure out all those constraints first". What happens then is a sort of writer's block - no matter what you end up thinking of, you constantly get distracted with caveats and gotchas, and rarely manage to have a session where you can fully "think through" subsystem A and draft a complete design. If this tendency isn't policed, you'll have a tendency in all of your "deep flow" sessions to have your mind start the session by thinking about the real thing you're trying to build, but then aimlessly wander off onto the paths of all the ways your thing interacts with other stuff. Don't get me wrong - it's a useful thing to be able to do, but it needs to be deployed "at will" rather than being a gravity well your mind gets pulled into by compulsion.
Ideally, even though you know some of it will violate some constraints, it's a hugely useful skill to be able to keep your mind on target and fully think through "subsystem A", any constraint-violations-be-damned. Sure, it won't work, but at least you'll have a mental sketch of the whole thing which can then be amended. Usually you'll find that the sketch is still mostly right, and more importantly, you'll now be able to conceptualize "the whole thing" instead of always getting stuck on the first third.
---
The skills to maintain - and reestablish this "mental target lock" seem IME to be identical with the ones you need to stay in a flow state even with people around you constantly distracting you. Even if no "other people" are distracting you, and you're in a private office, you'll still be distracting yourself, so — I've found that a bulk of the improvement has to come from you, yourself, developing mental discipline on this, rather than from you having a favorable environment. Otherwise you'll have a private office to just daydream about development, rather than constructively thinking through things.
A favorable environment can be great to have, but the ability to rapidly recover from distractions is something that makes a developer extremely effective even if they've got a favorable, external-distraction-free environment, rather than being a skill that becomes worthless if the external distractions are removed.
It’s quite possible; Microsoft perpetrated a brain-dead interpretation of TDD around the time the author is describing (2008) and there are still people there who believe that’s what TDD is.
(Their definition: write all the tests, then write all the code. Moronic.)
Particularly it's the testing done by other people and the unit testing. Testing people are certainly needed in many cases, but the article gave no reason why it has to be done by other people. Just because devs spent a lot of time working with tester, and they enjoy that time, does not make that way of testing the better choice. I worked with tester in Google, absolutely the most unproductive organization inside Google. All normal places have conducted their own testing before releasing to customers, internally most (had no experience working on consumer products).
Edit:
> Sounds like you had a bad tester. Opposite of my experience. A good tester is a gift from the Heavens.
This sounds like just a statement without backup.
Sounds like you had a bad tester. Opposite of my experience. A good tester is a gift from the Heavens.