Organizational Skills Beat Algorithmic Wizardry (2015)
johndcook.com
johndcook.com
It’s related to what I think of as “coherency” and I’m often surprised at how long it can take.
The most recent example is that I was writing up some technical feedback for a reviewer of a product I develop.
My first draft was several pages worth of dense prose. I realized I needed to simplify, otherwise it would overwhelm the reader. I worked on it for several hours, and in the end I had it down to about 4-5 paragraphs.
So it took me five hours to write what looks like a simple, straightforward email!
I’m a music producer by trade, and the principal is very much at work. To create a successful recording requires getting into the small details of the sounds, and working with them, whether your tools are complex or simple.
Like many others, I've been playing with digital recording tools off & on since the 90s as a total amateur (wanted to be in a 'cool band' since middle school, still do), and while I know there are a thousand great guides out there, I always seem to find them overwhelming/paralyzing.
Admittedly, I probably don't travel in the right circles for it, but the github and google docs aspects in particular sound like something I haven't read as much about musically inclined people doing before.
The answer is that I can write a competent blog article about as fast as I can type. Decent web copy that boils down the essential benefits and features of a product and expresses them concisely and compellingly takes much much longer to get right.
I came to conclusion that my ability to code on the micro scale, e.g. at the level of an individual method or algorithm, probably stopped improving after 2-3 years of professional experience. I think you could take the 24-year-old me, ask him to work on a narrow problem, and he'd do about as well as I would now.
But where the 24-year-old me struggled a lot was at the macro scale. He didn't know how to compose those individual pieces into a cohesive hole, and instead fell back on a handful of patterns like singletons. I remember horrible days trying to unravel my own code and figure out how component X was going to communicate with component Y, and the result was often a mess.
I think this macro-scale ability really does improve continuously over time, as you are exposed to more and better patterns for integrating components and managing dependencies.
To be very honest, being on Google searching constantly has been the most important skill. In this face paced IT environment, if you stay too long becoming an expert on something, it's replacement is coming right around the corner.... even within the same framework. As I was finally getting a hang off Spring 4 and Spring Boot 1.x, then comes Spring 5 and Spring Boot 2 with their new WebFlux architecture.
Equally, they should get their hands a little dirty. Best is to be a great chef but spend some time with the bottle washers relating to being a dishpig. Nobody really likes Gordon Ramsay as much as the chefs who invest in staff development.
I prefer Perl.
... do you realise how ridiculous this sounds?
Let me say that I get your point of view: I know of the existence of smart and educated people who are lazy, presumptious, have a superiority complex, talk down on everyone and/or are fakes.
Do you understand my point of view though? Are you aware that people exist who are less educated people and who can't grasp that valuable complex things exist which requiring specialized knowledge, and that those less educated people being unaware of the existence of such things automatically and wrongly assume that their specialists are fakes?
And know multiple people who are like he described: generally keep aura of looking smart are are good in few selected areas and generally don't really do much work. And they always get absurd amount of benefit of doubt the same way you do here. Meanwhile anyone who criticize them is assumed to be stupid hospital janitor who don't get that they are the ones who do real work.
And know, they do not do "valuable complex things" nobody else cant grasps on the project. They talk about complex things around water cooler to impress people.
And yes I do know people who do complex things too.
> That is not nearly accurate analogy. Note that doctors in hospital do more then just looking smart.
That's precesily my point. Doctors in hospital do more than just looking smart, even though janitors may or may not realise it. It's really an excellent analogy.
In the IT field it's different though. Every junior is absolutely sure they are better than their team lead (been there, done that, also been the team lead). Every QA thinks that programmers are lazy brats. Every PM too. Every programmer thinks QAs and PMs are stupid and can't grok their (programmers') work (which, so far I think that it's indeed true (maybe not the stupid part, but the 'can't' part), and that's why the industry suffers).
> I didn't know a janitor who thought that nurses or doctors didn't do valuable work (greatly more valuable than their own).
More than once I've heard staff (not janitors though) talking depreciatively about doctors (e.g. "he thinks he knows everything", "he said X but my uncle says Y").
I think if you read what I wrote a bit more charitably you'll see that my point wasn't really about janitors. I used them figuratively to comment on OP's attitude that seemed like the terrible combination that is confidence and ignorance.
Much harder, no doubt. More valuable though… a good portion of that value comes from having a clean hospital to begin with.
(Not that I am not talking about market value, which is obviously much lower than that of doctors, thanks to the larger supply of janitors, and how much cheaper it is to train them.)
This is not a great analogy. "Algorithmic wizardry" and "organizational skills" are two distinct skill sets, and it's perfectly possible to be great at the former but lousy at the latter.
I'll tell you one anecdote, when big data was starting to bubble up, there were PHD's that were hired at two firms I worked at. Eventually in the long run big data was done, but it was not from their efforts. It was the grunts.
Depends what the value adding function is. I work in a field (essentially CAD) where getting anything done takes years at best and if a technology does not have an estimated lifespan of at least a decade we don't invest in it.
Web stuff is of course starting to spill in.
You know the adage "Work smart not hard"? That's what you get a PhD for.
On Fridays I typically spend the afternoon deleting as much code as I can
Just because you don't have to write some novel algorithm doesn't mean you're not solving hard problems.
I'm currently in a very similar situation - a project in which we have a table of records, two of which are special in a certain, business-related way, but need to be presented along with the others, so to get around this we place "ifs" here and there - a lot of them.
It's a generally simple project that has been turned into something overly complicated because somebody lacked the foresight to design the architecture better.
Instead I am jumping from file to file, trying to work out what is going on, what all the seemingly unnecessary code is doing. It can be both tedious and difficult just to get data between the database and a webpage.
I want to rewrite a lot of it, but I understand that working, tested code is actually worth quite a lot, and there may be functionality that i haven't understood yet.
That idea - the distinction between tedious and hard - is going into my mental toolkit. Thank you.
This is from the preface of SICP[1] which taught me that abstraction is the most important concept of programming (and computer science?). Do you need to know how sqrt() has been implemented to being able to use it? no. A programmer shall first have a good knowledge of the abstraction techniques. He shall know how, when, and the cost to use them. Then, if you are illiterate when it come to algorithm design and analysis, it shall not be a problem because replacing a bad algorithm/implementation by another would be easy due to the correct application of abstraction. "The cleaner and nicer the program, the faster it's going to run. And if it doesn't, it'll be easy to make it fast." — Joshua Bloch
These were my top 2 surprises when I started working as a programmer:
https://henrikwarne.com/2012/08/22/top-5-surprises-when-star...
This really hit home for me. I personally find it very difficult to get emotionally invested and therefore effective when working on a codebase that I don't personally like. But that's not very professional, so I tend to phone it in when in that situation, at the expense of really using my talents or dedication. But I know that other professional coders do better than me and somehow find a way to be very interested or invested in things they don't care about, perhaps because for many coders, the code itself is enough to warrant the personal interest, and less so what the actual code is used for? If I don't have a big-picture appreciation of a project, I don't enjoy the small stuff either. Perhaps that means I'm in the wrong profession.
Additionally, most of us are limited in what we can learn in a lifetime. Talent is far from the primary limitation (most people will be average in terms of innate ability; my guess). Far more important is the time available to study and learn new skills and the motivation to do it. Then there are issues of personal preference: if you just love algorithms, you will tend to focus on them, if you love building systems, etc.
There are many, many more opportunities for organised people who aren't great with algorithms, than disorganised algorithm geniuses.
If that is how it works, I did not really experienced it. In one place where they attempted this, senior ended completely out of touch with his architecture problems - because it was only juniors experiencing them and never him. In big corporations, architect does super uber high level and negotiations.
Big refactoring and architectural changes were done by seniors and require experience, of course. But that is only part of what senior does, a lot of if is using existing architecture. Also, juniors can grow into such senior only by taking on larger and larger tasks - you don't get there by filling small boxes without doing own decisions, mistakes and living with them.
The biggest problems, especially in larger projects, are organizational one. As in, algorithms part is what is often done by juniors, who often actually can do them well being just out of school and being trained in them lately. And also who have bigger uninterrupted chunks of time then seniors.
There are algorithms for sorting with known properties. There are no algorithms for problem analysis - just a lot of tradition (e.g. “refactoring”) which is hand-wavey even when it’s done well, and actually adds complexity when done badly.
The kind of problem segmentation => simplest solution we’re talking about isn’t even mentioned on most CS courses, and certainly isn’t taught in any useful way - possibly because it hasn’t been researched or understood.
My favorite is to have someone explain to me what happens when a browser makes a HTTP request to a web URL, which has <insert framework and DB here>. Network guys will tell you all about the request and response, web developers will focus on the front end, devOps will tell you how all the pieces communicate, etc. My expectation is for someone to at least be able to give a high level of all those things, while also having detailed knowledge of some specific areas... and seeing how they choose to answer such an open-ended question tells you quite a bit about where their focus and attention lies.
Just look at Sandi Metz's or Avdi Grimm's or Martin Fowler's writings and there's lots of good ideas on how to organize your code.
This is insane, what is imp? Giving too much info or not, asking more questions or not?
Oh man, sometimes I hate this process of interviewing.
- how are you as a person, a coworker
- are you able to ask for help?
- can you hold a conversation?
- how do you learn?
- describe a failure in your career and how it went from there
all this is for the interviewer and the interviewee to get to know each other each other and see if the candidate is a good fit for the company culture and values. If the recruitment lets in a narcissist or toxic person it will poison the well.
They do. All the time. More often than you'd think about something as trivial as "can you actually write code".
And so, you'll need to demonstrate your skills. Find a better way with an extremely low false positive rate, and you can make a fortune consulting and implementing that. It's not like anybody likes doing these interviews.
> And so, you'll need to demonstrate your skills
It is the "really hard" part that is criticized here, not the "algorithm" or "code something" part.
That being said, asking logical puzzles is fine when you are hiring juniors you expect to train. The question with them is "is this kid smart enough to understand what I will teach" not "does this kid already knows everything". However even there, really hard algorithm is not something I would go for.
Really? That's not how I read it. And when I asked OP, that's not how he responded. The examples of questions that he said he would ask had been stripped of any algorithm-related questions, or any questions that involve coding.
Then you said "it's really hard bit that's being criticized here, not the algorithm bit."
No, it's the algorithm bit that serpix was originally criticizing, as evidenced by the fact that when he replied to me he didn't include algorithm questions from the list of things that he would ask.
"asking logical puzzles is fine"
What you want is a question that has multiple answers, that's dirt easy, and where you can ramp up the difficulty to the moon if the candidate breezes through. These questions are hard to find, but they exist. They require investing some thought, so they're unfortunately not asked too often. But they work. (For small values of work. There is no good interview process)
[1] Somewhat hard. It's solvable in 45 minutes, it's not that hard.
I also don't find any value in testing "How someone solves a problem," since people solve things differently. I have no guarantee that my way is the best, so why would I reject someone else just because their brain works differently than mine does?
No, because organizational skill involves also following (not required for algorithmic wizardry):
* Solving same situation consistently the same way.
* Differentiating between important for maintenance and unimportant.
* Ability to plan - think in advance about how much time people need to answer your questions and giving them time. Realize that other people are not available at your whim.
* Negotiation, understanding what other people mean and being able to express what you mean the way others understand.
* Being able to do also boring parts, not just fun stuff. Keeping up work even if nobody controls you.
* Being able to know what is "good enough". Organize work the way that most important parts are done first, don't just follow what is fun.
* Writing the code the way other people not just understand it. So that tests are possible and actually write tests.
There are many people, algorithmic wizards or not, who cant do the above. That is why agile have standups and jira tasks with max 0.5 day size by the way, because way too many people never finish tasks they are assigned to without being checked on every day.