Knowledge workers are bad at working – and what to do about it

As a programmer, one of the most effective forms of deliberate practice I've ever experienced is to write a complicated piece of software, and then completely re-write it from scratch with a different approach (and then do it again).

Of course, that almost never flies in the business world. They'll ask you why on Earth the company should pay to build something again before they make money from it. However, it truly is a powerful engineering process for many types of problems.

The point is not to merely rebuild it for arbitrary reasons. Rather, to take stock of the benefits and the pain that arises from the current design, and to try completely different approaches. You do it because there's a nagging voice in the back of your head that tells you that you've left some "goodness" on the table ... that even though your current implementation "works", the problem could be solved better.

You've already "loaded the problem into your head," because you've solved it once. The second solution will improve in some ways and possibly regress in some ways. Often, the 3rd or 4th solution will be "right" -- which means, not perfect, but significantly better than your earlier approaches.

It seems kind of like learning the card game of Bridge by playing the same hand over and over to see if you can get it better. Or learning chess by starting from the same canned starting position and seeing if you can play it to completion better.

Some context: Here Cal Newport is extending some of the ideas presented in his latest book "So Good They Can't Ignore You".

In the book he says (among other things) that how you become 'so good that...' etc is by what he calls 'deliberate practice' - which in turn has two components -

(1) 'deep work', the doing of which actively stretches you while you are doing it (in term of guitar playing, this would be acquiring new rhythmic abilities, vs just playing/performing songs you already know). The 'practice' in deliberate practice should maximise 'deep work' (as per Newport)

(2) fast feedback loops - either by other (and ideally better than you) people, or built in (if you play a note wrong, you know instantly)

This blog entry and its neighbors seem to explore how he applies this philosophy to his chosen activity (research in CS). It is not entirely clear how to apply it to programming.

> In order to achieve consistent deep work, I usually need to first immerse myself in the relevant research literature, seeking a problem that it is recognized as important, unanswered, but probably answerable with my skill set. This is not easy.

I want to run out and repeat this to every PhD student I've met in the last 2 years.

The claim that knowledge workers spend all day answering email and do nothing to focus on self-improvement is just plain bullshit. The fact that we're presently using a website that essentially serves as a crowdsourced, real-time trade journal counters that argument. The existence and wild success of engineering-focused conferences counters that argument. The proliferation of easy-to-access information for professional growth allows for a self-selecting group of people to improve their lot in life just by picking up some new skills and getting some experience with them. You can learn a new skill literally by browsing the Internet these days. As such, I don't really view the knowledge worker as a person at risk of stagnation without actually making an effort to be lazy.

There are some people who feel that they already know everything they need to know, and don't want to better themselves. It is entirely attitudinal, and in no way unique to the "knowledge worker" classification. I would argue the problem is far more significant in other industries, such as manufacturing and other heavy industry. When a specific kind of job replaced by a machine, or an entire industry gets outsourced to another country, the individuals previously responsible for that kind of work are likely not to have been trained for any other function. That would probably be a better audience for this kind of thing, compared to people who spend their free time reading about new technologies and playing with them on the weekends for fun.

On a related note, there is a very real risk in larger, more bureaucratic, organizations that it becomes easier to be recognized for shallow work than for deep work, managers like metrics and the shallow work is typically very much easier to quantify - leading to the Dilbert "I'm gonna right me a new mini-van" scenario - or whatever the social media equivalent is.

I like the idea of making sure your tasks have a certain amount of "stretch" (i.e. push yourself a little with every task you give yourself).

We focus so much on our skill level that we forget that the "flow" state requires challenging yourself: https://en.wikipedia.org/wiki/Flow_(psychology)#Conditions_f...

I agree with the OP. My day job is somewhat interesting but I have a feeling once I've figured out how it works that I'll be back in the same boat of doing less of my craft and more process. I am very lucky that in software engineering the body of knowledge is wide and always changing and that I genuinely like it.

At night I've become more accustomed to reading tech books on the newest technology instead of whatever book for book club. Our profession almost forces us to keep up or get left behind. I love it. Always gets me motivated to think and expand my knowledge.

This is a great article, I can totally relate. I have been doing work that had not involved deep work for maybe a year, I kinda had plateaued my learning as a result of an unchallenging and demotivating work environment, I realised this is not what 'I' wanted to do, so I got out. I have hacker news to thank for this partly too.

4 months later I feel like a kid again my mind is like a sponge for knowledge, and because I have cleared my desk of non beneficial work, and as a result I have been learning at a fantastic rate.

Things I have been spending my team on include, functional programming techniques, JavaScript client architecture, pure REST api design, algorithms, visualisations in the browser (d3), geojson, node, mongo, redis, to name but a few

I think this article is a great summary on how to improve your skills as a knowledge worker. I see people that believe their mental capabilities will never improve. It really comes down to step #3: the stretch. Many people skip over "the stretch" in all areas of life and reach "steady state" relatively early on.

Please spare us this trite bullshit. I see people trying to justify their inability to improve themselves all the time. "If I'm having trouble, everyone else must be too!", they think, instead of considering the obvious: maybe they just suck.

There will always be time-wasters and people who get off on bureaucracy. But I'm willing to bet many of us, especially here on HN, routinely get admonished for "forgetting" to do menial, time wasting tasks in favor of doing what we love. If you're wasting your time instead of taking every opportunity to learn you'll get no sympathy from me.