What I Wish I Knew When Starting Out as a Software Developer: Slow the Fuck Down
blog.salsitasoft.com
blog.salsitasoft.com
What you don't learn in school is how to write maintainable code. This is a much different style of writing code, and much more valuable once you go out in the "Real World". I'm not sure if there are CS programs out that start with a small project and then force the students to make massive changes to that project throughout the course of the year, but to me there is immense value to that. Even better would be if they had CS101, CS201, CS301 and the students taking CS201 would be forced to dig up their project from CS101 and work on that, CS301 would work on the code from CS201, etc. It's eye-opening how much you forget about your code after a few weeks, so it would drive home the notion of commenting, writing easily readable code, etc.
Saying: "Use the same thing you worked on last quarter" dooms everyone who made wrong foundational choices with their old project to spend most of their time "fighting the last war".
Not unless you told them before they started the previous year's project that it'll be used further along the line.
I'd just make it one really long subject that spans 3/4 years.
If the students are being graded on a curve so they're competing against one another, this would be downright diabolical.
In the past few years, I've worked repeatedly with existing code bases to extend or modify functionality (or just fix problems), and it has been an amazing learning experience. Not all of the code has been good, much of it has been questionable. But the process of understanding someone else's code, figuring out how to change it, balancing the harm of forking vs the benefit of improving, dealing with pull requests and merges… there are a number of skills you will only learn from years in this kind of environment.
One huge challenge here is when you don't work in an organization that understands software. People aren't knowledgeable about software are often impressed with green field programmers "jim wrote the whole thing". Truth is, understanding a different code base and getting to the point where you can improve it and contribute back to it is (at least in my experience) far more challenging than just writing something from scratch. It seems that most people who write software are already well aware of this, but it can be a hard sell outside (in fact, you may even find that people seriously question your progress - a green field app there's something nifty and new to demo even day, with existing code bases, there's a lot of confusion and investigation - and it can be very dispiriting to spend days stubbornly trying to understand and diagnose a feature, finally solve it, and realize that the team is disappointed with your progress).
However, the good software teams appear to be well aware of the challenges, and it's an essential learning experience.
For example, the conventional wisdom that functions should be less than a page of text and that they should do one thing and as abstracted as possible actually makes it quite hard to understand what a program does when you aren't familiar with it.
Instead, I'd propose first writing straight line code, and then abstracting out the common functionality into separate functions as necessary. Casey Muratori (the guy behind Handmade Hero) calls this compression oriented programming [1].
It does? Maybe you mean something more extreme when you say "as abstracted as possible" but for /reading code/, I generally find it easier to get the high-level picture and avoid getting bogged down in minute from code that looks like this:
function sendReport(params) {
var dataSource = findDataSource(params);
var data = dataSource.generateData();
var report = reportBuilder.build(params);
sendReportEmail(report, param.recipients, this.template);
}
... versus a flatter one-page implementation with 15 lines of data loading, 20 lines of report, 5 lines of email templating and 15 lines of communicating with the mail server... all sprinkled with variously scoped try..catch blocks. It also helps me focus in on the part of the task I'm really interested in without trying to load the entire system in my head.For /writing code/, I do agree the reverse is often best: start by doing the simplest thing that works, and then refactor it into something cleaner.
function do_task() {
do_step1();
do_step2();
do_step3();
}
function do_step {
step1();
}
function do_step2 {
step2();
}
function do_step3 {
step3();
}
But I've rarely seen that be the problem. It's been far, far more common to see giant functions that are much more understandable when broken up.Usually I am only interested in specific parts at any given time, either because I only have a particular task to accomplish (e.g. update the email template) or because trying to understand everyone at once would overload my little monkey brain.
If I'm trying to just update the email template then I don't care about the part the loads the data and I'd be wasting my time reading it. Sometimes you can't even tell if an chunk of inline code is relevant without spending a decent bit of energy figuring out what it's doing. Having everything in a single function also makes it easier for lines of unrelated code to become entangled together and thus harder to understand.
Well-structured code gives me a choice; I dive in and see the details if I'm interested, or I can leave it for another day and focus my attention elsewhere.
Certainly though, good tools like IDE are vitally helpful in reducing the friction in peeling back the abstractions when necessary. The nature of some languages though means that the level of tool support available can vary widely, which probably has an effect on different prefered coding styles. In Java, for instance, it's pretty easy to find all the pieces of code that call a particular method, while in Python, it's often not really possible without running the code.
Also, I probably should have been clearer above. Sometimes a short function is the right approach, particularly for something that is going to be done over and over again, but the overall goal is clarity, not short functions.
Anyways, it's not worth getting too hung up on I don't think, it seems my opinion is the minority one.
I've never seen this discussion play out where the "short function" side is saying that the only goal of short functions is...short functions.
Obviously "clarity" vs "short-functions" is a no-brainer discussion. But fails to acknowledge the complexities and nuances being discussed.
Shouldn't that 10 line function be pretty much self documenting?
Done right this should really help get a good grasp of code base quickly.
Now, if the function has bunch of side effects and you HAVE to read it to make sense of what's going on one abstraction higher, then it is done wrong.
This hasn't been my experience at all. Functions as small as 100 lines can absolutely destroy my comprehension to the point where I need to refactor it into smaller functions to be able to understand it. I'm dealing with one such function (at 200 lines) right now at work.
Well, let me rephrase. I can see what it's doing easily enough. I just have no idea what it's supposed to be doing, on a chunk by chunk basis, because it's all hand rolled search loops and array manipulation.
I also know the final result is incorrect.
I'm sure I'll slap my forehead and wonder why I couldn't see the problem before after I'm done refactoring it.
I, however, do it differently. I start from a high-level of abstraction, and work my way down. At the top level I might have a function called "ImportData". I write that call down, then move to the next line where I write "RunDataAnalysis". At that point, if I decide to move down and implement the second function, I forget about what ImportData does, how it does it, or whether or not I've implemented it.
All I know is that the data structures are populated with the "Imported" data, and proceed to code along with that assumption. Obviously if I test, it won't work, but that's for unit-tests. You do unit-tests, right? Can't do that meaningfully with giant functions, but I digress.
To me, functions are a form of "interface". It delineates responsibility and expectations between different bits of code. I'm not a chaotic and/or artistic programmer and I'm definitely not a genius wunder-programmer that can hold the entire functioning of the program I'm writing in my head at any time. What I can do is systematically break-down a problem into component pieces, negotiate their interaction, and make it solve a problem.
Setting a reasonable pace with the project could be tricky, but I think it would be very beneficial.
Internet time does not justify poor program design.
It was the first time I realized that bugs in earlier assignments don't magically disappear, and changed my thinking dramatically.
I studied Software Engineering at Swinburne University in Melbourne Australia, and we did exactly that.
One subject was "Software Maintenance Project" where we took an existing code-base we didn't write (a front-end for laTex for our year, the CVS code base for the next year students) and continually made changes to it. Halfway through the course we had to deal with changing requirements. We were graded on the quality of our code and how well it fit the existing style etc., not how fast we did it.
We also did a full-year final year project with 16 of us where we created a BIG code base from scratch. We decided to use an iterative approach (our choice) and so over the year we ran 6 iterations continually building on what we'd already done.
We also did awesome courses like "Personal Software Process" to teach us discipline in how we personally approach writing code.
EDIT: Sometimes I post to HN "hire/resume" threads about my hire-ability and how I personally think I'm "better" than the average person who studied CS because of things like the above. Normally everyone disagrees and says as long as I can code, it doesn't matter that I studied Software Engineering and am a certified Engineer. It's for reasons like the above I still personally feel I'm stronger than the average CS grad.
My one piece of advice: keep studying and practicing making beautiful software, in and out. Be careful who you take technical advice from; the industry right now is flush with cash, and thus there is a huge incentive to talk more than produce. Projects are often killed as quickly as they're started, causing people not to practice program design.
You had the option of building them all completely separately, but you were strongly encouraged to re-use code form earlier revisions, particular for the client-server bits. The class also included labs on Subversion, although nothing on the notion of things like code review. I could imagine teaching the course today with Github and requiring (or at least awarding some credit for) good use of Pull Requests, meaningful code review, etc.
Sadly it looks like the course no longer exists; I have no idea what replaced it.
What I have found is that you need a balance. "Slow the Fuck Down" gets you working software, but if that takes you twelve months to a year, you lose the sense of progress and accomplishment that having a live product gives you. There has to be a healthy balance between "Slow the Fuck Down" and "Get Shit Done" which usually involves cutting features, prioritizing and temporary fixes. Most projects using MOSCOW is a good place to start with this.
Also, what is MOSCOW?
Somebody somewhere said that:
"Estimate the time it takes to finish the project. Then multiply that by 8"
Surprisingly, this usually lands in the correct estimate. Best advice ever.
Yeah, I know that it's not likely to work...
It doesn't make sense in the long run to do this "fix, patch, fix, patch" non-sense .. but still companies need to ship shit to some deadline, so we do it.
80% of workplaces are like that.
I think its the incentives of billable hours that does this. They want low estimates, so they can win the work at all costs. They have to do it because other companies also underestimate. That or they think pressure motivates developers, but it really results in terrible code.
I personally advocate using accurate estimates, then applying a discount if you have to.
Underestimating is very common with programmers, yeah, because they think they can do it in the given time, but in reality that almost never happens, at least to my experience.
What is an accurate estimate ? It changes all the time when new pitfalls and problems are discovered during the development cycle. So accurate estimate is something that almost all the time more than the original estimate, if you think logically. So better to just multiply by some factor that is realistic.
But I agree with everything you're saying, which is why I try to get into more test centric cultures when job hunting.
Been there, tried that. I make it as easy as "run this bat file to test everything" in this specific sub-project. And then I watch them make some silly little "test-program" to debug the thing. Not only that, but we discuss a feature in said sub-project that they're implement and I hear disturbing comments such as "I hope it works" or "I hope it all still works after I'm done".
Part of our jobs as professional developers is to understand the potential business impact on a feature and not invest time on documentation or unit tests for code that will probably be dropped next week. That said, and not to be harsh—but if you want to be treated like a professional, act like one, and set expectations for how much a feature costs to deliver, rather than breaking your work down into separate tasks and inviting your supervisor to discard the tasks he doesn't understand.
To fix the corporate world is to say "No" more often.
The best thing you can do in this situation, is hold your ground and slow the fuck down. Not because you're a recalcitrant bastard, but because you are the one responsible for quality. Once you learn this, and really internalise it, a lot of things fall into place. Projects are a tug of war over the time / cost / quality triangle. Make sure you're pulling your corner.
Eventually some of the management may learn how unwise this is, but it can take a long time, especially when many projects don't end up being successful/used by intended users anyway (which is its own drag on morale). I don't work in this environment anymore, thankfully.
Maybe it's because I've worked in smaller teams but there was no real pressure to deliver a month earlier. When we presented to other people in the company they don't really care if we covered 38% of cases this month or 42%, they simply don't know what those numbers mean.
> Peter asked Zawinski, “Overengineering seems to be a pet peeve of yours.”
> “Yeah,” he says, “At the end of the day, ship the fucking thing! It’s great to rewrite your code and make it cleaner and by the third time it’ll actually be pretty. But that’s not the point—you’re not here to write code; you’re here to ship products.”
> My hero.
The more important lesson I would impart upon my younger self is that the code isn't done until it's been committed, there's been a README written, and some testing has been done. Who cares if it works? Pft, that's the easy part!
If your commit messages after the first few aren't more specific than 'updated' or 'new stuff', you fail. If the README is 1 line long and just says 'I made a thing', you fail more.
And then there's this:
You dig up old code, and it really sucks. The idiot that wrote it should not be allowed behind a keyboard! Y'all know the end to this story - the 'blame' command points the finger... at yourself.
There's nothing more humbling than that.
Proper software engineering isn't about writing code, it's about maintaining developer sanity, especially in the face of fast-moving business requirements. Don't push back hard enough and the sale's guy's unrealistic schedules start to dominate. Push back too far and you start writing a frameworks on top of frameworks instead of deliverable code. Push back too far and you end up with lazy engineers who drop features because they're "too hard" to test (Hello Google, so glad you decided not to drop ext4 support in ChromeOS's Files app.)
But in today's fast-moving world? Run. Run as fast as you can. Keep running.
Ouch!!
A good automated test suite has a very high (>90%) coverage of all cases that an actual user may encounter in the real world. It is extremely difficult to achieve this in the initial version of the test suite. The goal initially should be to build a test suite that can be quickly and easily updated so that >90% test coverage can be achieved over time.
When people are "using and verifying software" or when there is a bug found in production, the bug fix should be accompanied by an update to the test suite. This way this particular bug will never happen again in production. In addition, you never let your guard down and continue this process for as long as the product exists. Over time this process will yield a test suite so good that you can assume that a particular change hasn't broken existing functionality even though there isn't a 100% guarantee.
The cool thing now is that build tools are so good that you can codify some of this process into the build to ensure commits are tested before merged. And commits are reviewed to verify relevant tests were added or updated before merged.
It can be hard in environments where even hours longer on a piece of work can make the difference between happy customers and sad ones but it pays off quite quickly.
You write better code which is easier to maintain and has more tests. In doing so you are able to improve your craft which is hard to do when you are rushing yourself.
Once you get better you will be more productive and can get work done faster but with acceptable quality.
Over-performance will never be rewarded. One might get a slightly higher bonus if one is lucky, but generally it'll be forgotten. All those long days, nights, and weekends spent working non-stop will be immediately erased from the company's recollection once the product ships. They will be especially forgotten once the developer can no longer sustain that pace which now has become the norm and needs to be reprimanded (despite still being objectively productive).
In the end, it's only the developer who suffers, and it'll be great suffering up to and including mental and physical disability from not slowing down. Excellent advice.
This means we need to step up as engineers and start becoming managers.
But we also need to refuse to work for managers who are incompetent.
IF you're a programmer and you're reporting to someone who is not the CEO and is not an engineer, then you need to find another job and tell them why.
Time to stop working for startups run by CEOs who have nothing more than a business degree. IF the CTO is a competent engineer and the CEO has some domain expertise that's fine.
At amazon my boss, literally, had wanted to be a prison guard and gone to school for that. He was a terrible manager. They didn't even hire engineers to be managers, they just got these random incompetents. His boss was right out of the DMV. It was that way all the way up the chain of command to bezos. When nobody between you and the CEO, including the CEO, is a competent engineer, you're not going to be managed well. (and Amazon's software quality reflects that, despite the constant hype the site is a tragedy of bad design.)
I'm not so sure, you want engineers to be engineering. You want good managers to be managing. The skillsets don't have much overlap and people who are good at one might be lousy at the other.
It's easy to learn to manage programmers, I've done it, and I did it by dealign with a lot of bad managers.
In 25 years I've yet to meet a manager who was good at managing software development who did not have an engineering background. At best, the best managers sat by and let the programmers run the show and they themselves just helped with interpersonal problems and hr, and spend their time doing some project management type efforts.
The idea that managers possess something magical that engineers don't have is false.
Or maybe over decades I've just randomly never seen it. And I've worked at places like Amazon and Microsoft-- at amazon the managers were not engineers and it really sucked. At microsoft the mangers were engineers and at least in that regard it was much better.... except at the higher levels where it became political and the mangers stopped being engineers, and thus projects got behind.
Managers who only manage people have no clue about what it takes to think technically, so they tend just to apply their human management techniques to a technical AND human problem. Engineers can become managers, but manager only people cannot become engineers during the course of leading a project.
Most obviously, Michael urges young programmers deciding how much effort to put into their work to "ebb towards underperformance".
That's not what I said. From the source (on Quora, here: http://www.quora.com/What-do-software-developers-age-30-and-...):
3. Recognize underperformance and overperformance and avoid them. There are a lot of low-effort players who stay employed for years. This isn't a bad strategy if you're settled, but I wouldn't fall too low. That said, the only people who typically get fired for underperformance are the people who fail so badly that they generate work for others. People who hide and do little tend not to make any enemies. At the same time, be cautious of overperformance. This isn't like college where challenging your professor's ideas could earn you an 'A' if you argued your point well. Overperformers often also generate extra work for their superiors and colleagues and draw unwanted attention (see: McNulty in The Wire) and are more likely to be culled for "performance" (98 percent of "performance management" in companies is politics) than underperformers. I'm not saying that you shouldn't work hard and do a good job and learn as much as you can. That's not necessarily overperformance. In my experience, though, overperformance is much more dangerous than underperformance. It can get you just as fired and it will happen a lot faster. If you end up stuck between the two, prefer underperformance.
I'm not saying that people shouldn't work hard or do a good job. On the contrary, I think young people should bust their asses at growing the skills and contacts that will serve them well for the rest of their lives. I'm saying "avoid overperformance", and overperformance isn't the same thing as "doing a great job". I'm saying, "don't become like McNulty on The Wire" (or a certain person at Google who tried to prevent a product failure and got burned for it). McNulty is a textbook overperformer: he loves the job, it doesn't love him back, and he puts so much into it that it destroys his life and career.
Sorry, but I'm pretty fucking pissed out this being out there like that. That was way the fuck out of context. I wasn't happy with the Lifehacker edit either but I didn't speak up about that because I didn't see it as a big deal. But I am definitely not telling a generation of people to slack off at their jobs.
As a previous overperfomer(who got burned for it) I find overperforming is often a product of misunderstanding what the company wants you to do, and in the end it doesn't help you, your peers, or anyone except your personal idea of what you think is right, without regard to reality.
To quote from Ribbon Farm article: "The simple reason is that if you over-perform at the Loser level, it is clear that you are an idiot. You’ve already made a bad bargain, and now you’re delivering more value than you need to, making your bargain even worse. Unless you very quickly demonstrate that you know your own value by successfully negotiating more money and/or power, you are marked out as an exploitable clueless Loser. "
[1] http://www.ribbonfarm.com/2009/10/07/the-gervais-principle-o...
Can someone explain this formulation of 'performance' without referencing "the Wire"? Ie is it simply long hours? Or is it long hours coupled with getting a lot done, in a good way? Or is it getting a lot done, and breaking things for everyone else? Or ...?
For example if you are a developer and realise your project needs to engage another team NOW for a critical deliverable due in 3 months and you spend your time on this task then you run the serious risk of pissing off both your manager and the project manager.
Or for example putting together a prototype that shows how another teams work could be automated.
Or pointing out flaws/discrepancies of a high profile product. Or telling a senior manager how a project is really doing, etc.
The bottom line doesn't matter as much in terms of respect and career progress as it should. There is a misconception that you can get away from politics if you work for startups or smaller firms but that's just not the case.
EDIT: That said, I do agree with the post, as someone who is also twenty years in.
This case from yesterday: http://fuckinggodateformat.com/ is slightly better. It tells the reader that the author of the page doesn't like the go date format in a succinct fashion. Though, on the other hand, it primes you for a rant rather than a cheat-sheet.
This is one reason I don't use swear words in my blog posts. Also I don't my site blocked by any filters. My personal policy, but I really don't care if other people do it.
I've observed that programmers from successive generations become more and more likely to just sit down and start writing before they've even had proper time to think.
I think there's a spectrum, with one extreme example being the person who writes 1,000 lines in 2 hours of code that is buggy and unmaintainable but that the boss is really impressed because it does %70 of what is needed. These guys really impress PHBs.
At the other end of the spectrum is the guy who gets the same assignment, does nothing but sit in his cubicle thinking for a week. And then over the course of a day writes 100-200 lines that does %90 of the assignment, and with no bugs.
Current culture seems to favor the former guy-- which is why we have unit tests (to catch all his bugs) and the assumption that code is crap. There isn't much cultural support for the latter guy... but those guys, in my experience are the better programmers.
For trivial programs speed is king, but trivial programs can be done by either guy with a difference of a few hours in time to completion.
For programs of significance... the slower guy can save you months or years and even cut your required engineering headcount.
But that guy doesn't seem to be recognized for what he does-- in fact, he's often perceived as not working as hard, and is many times interrupted for no good reason, but because they think he needs more work.
> the person who gets the same assignment, does nothing but sit in his cubicle thinking for a week. And then over the course of a day writes 100-200 lines that does %90 of the assignment, and with no bugs.
I think you're leaving out the person who writes the buggy and unmaintainable version in 2 hours, in order to discover any pitfalls they failed to imagine before starting the project, then spends the remaining 38 hours of the week thinking about the proper solution to both the foreseen and unforeseen problems.