"Smart and Gets Things Done" is necessary, but not sufficient
github.com
github.com
I've been interviewing lately, both at hardware and software companies. At microprocessor startups, I ask about the pedigree (past experience, not educational credentials) of pretty much everyone I meet. I don't do that at software companies. The reason is, you pretty much never hear about a new team successfully making a high-performance microprocessor. Apple bought PA Semi, which had a moderately successful exit, but PA Semi was basically the SiByte team, which left after SiByte was acquired by Broadcom, and SiByte was composed of key people from DEC who had been working together for over a decade. When you hear about a new team where most of the people are smart new grads, they usually spend ~ $100M over five or six years years, find that they don't have a competitive product (or, more likely, don't even have anything that's close to working). If they have funding left to burn they may pivot a couple times over the course of a couple years before they burn through their cash and implode.
And that's despite microprocessor design being close to pure reason, in the grand scheme of things. Experience and wisdom matter a lot more in most endeavors. Something you see a lot, even here on HN, when non-CS/programming/engineering topics make it to the front page, are people who try to extrapolate from a few "obvious" facts and then come up with a conclusion that's completely and totally wrong.
In software, you hear about successful companies founded by people just out of school, or even people who have dropped out of school, all the time. "Smart and Gets Things Done" can be good enough to make a decent product. But, something you'd never want to hear from a plumber or a carpenter is "well, I've read some books on the topic, and tried out some tools at home depot, plus I'm smart and good at hacking".
Things like carpentry, manufacturing, etc. are hard. They're done so well here (in developed countries) that it's easy to forget an idea of how hard they really are. If you want to get an idea of how hard they are to learn from scratch, consider South Korea after WII. Its GDP per capita was lower than Ghana, Kenya, and just barely above the Congo[1]. For various reasons, the new regime didn't have much baggage, and they wanted Korea to become a first-world nation. The story I've heard is that the government started by subsidizing concrete. After many years making concrete, they wanted to move up the manufacturing chain and start making ships (among other things). They pulled some of their best people who had experience in business (having learned skills like management, operations, etc.) from working in concrete to try to build ships. They knew they didn't have the expertise to do it themselves, so they contracted out the plans and got detailed instructions how to build ships. They got plans from Scotland, because Scotland has a long history of shipbuilding. Makes sense, right? But, when the Koreans tried to build ships with Scottish plans and detailed step-by-step directions, the result was two ship halves that didn't quite fit together and sunk when assembled. For historical and geographic reasons, Scotland's shipyards weren't full-sized, and they built their ships in two halves and then assembled them. Worked fine for them, because they'd be doing it at scale since the 1800s, and had world renowned expertise by the 1900s[2]. The Koreans eventually managed to start a shipbuilding industry by hiring foreign companies to come and, build ships locally, and show people how it's done. It took decades to what we would consider basic manufacturing working smoothly, even though all of the requisite knowledge existed in books, was taught in university courses, and could be had from experts for a small fee. "Smart and Gets Things Done" goes so much further in programming than it does in virtually any other non-academic field.
Today, anyone with a CS 101 background can take Geoffrey Hinton's course on neural networks and deep learning[3] and start applying state of the art machine learning techniques[4] to research grade problems within a couple months. But, if you want to build a ship, and you "only" have a decade of experience with carpentry, milling, metalworking, etc., well, good luck. You're going to need it.
[1] http://www.nationmaster.com/graph/eco_gdp_per_cap_in_195-eco...
[2] In 1913, 20% of the world's ships were built at Clyde.
[3] https://www.coursera.org/course/neuralnets
[4] About 1/3rd of the way through the course, he talks about a new technique that was published after the course actually started. The future of education is going to be awesome.
I think it's more about the technical difficulty of the project you undertake rather than its form (software or hardware).
Many startups are technically very simple.
[Note the caveats: obviously history provides us with examples like Linux of OSes from scratch which did get from zero to competitive in five years but that had rather more people working on it.]
The knowledge required to write device drivers or understand and manipulate kernel behavior is totally distinct from what is typically taught in Computer Architecture/Computer Science and Software Engineering, a massive amount of knowledge is required that I could see no way to acquire short of trying to build your way up designing/reading kernels of greater complexity or being interested in some field where consecutively deeper levels of knowledge are an exponentially more profitable use of your time such as information security.
The documentation on ALL modern kernel's current implementation details is sparse (yes, even Linux) and the knowledge is assumed which gives some truth to the statement "You just hack on it" in the sense that there's no way to learn it save actually doing it. In truth however, to really get anywhere you MUST know x86 (Or whatever architecture you are developing for, additionally, if you think this will be easy because you coded MIPS or SPARC in school, be prepared to feel dumb), you MUST know deep C and a slew of other topics that the typical CS education leaves you remarkably ill-equipped to even get started.
I'm in telecom semi and I know companies like Huawei are using a model where they have 100s of engineers in China with less than one year experience with senior engineers in other countries teaching them.
look forward to catching up later this month! :)
You were able to think of an edge case for language feature that neither of your friends had considered. Awesome. Smart. Realising that upfront is the kind of skill that saves you hundreds (anecdotally) of hours of debugging because you solve it in the design phase.
Saying "smart interferes with gets things done" is exactly what Joel was saying when he tacked on "and gets things done" - so you can't really use that to argue he left something out.
They would rather mull over something academic about a problem rather than ship on time. These kind of people can be identified because they love to point out the theoretical similarity between two widely divergent concepts. For example, they will say "Spreadsheets are really just a special case of programming language" and then go off for a week and write a thrilling, brilliant white paper about the theoretical computational linguistic attributes of a spreadsheet as a programming language. Smart, but not useful.
So had I been left to my own devices, I might have written some Ruby code that assumed it was perfectly safe to return a proc from a method. This is a real danger for me, as I like programming with combinators. And I would have been bitten by my mistaken beliefs.
The only thing I'll say in my own defence is that having made this mistake many times, I had the good sense to check how things actually worked when I got home. I used to know, I'd forgotten, and now I know again.
I don't know if that's "getting things done." It may be spinning wheels. Perhaps someone "getting things done" never wastes any time worrying about the difference because they never bother to wonder what happens when you return a proc from a method.
They simply avoid the problem in the first place, so not knowing is irrelevant.
Sounds like you fit both the criteria for "Smart and get things done".
I like programming with combinators -> They simply avoid the problem in the first place, so not knowing is irrelevant.
I think that a general awareness of the tools available to you is extremely useful. Specific knowledge isn't unless you're using them. Given that your friends "read about it today", I assume they don't often make use of this particular language feature - in which case it's fine not to know. In your case, not knowing was dangerous, and when you realised the limitations of your knowledge (unknown unknowns and all that), you corrected it.
My point is that I think all of this is covered in Joel's original essay, so your title isn't really justified.
Finally, I think there's a good essay to be written with a similar title, where you explore the point that "sometimes you need an expert". luu touches on this wrt hardware engineers: http://news.ycombinator.com/item?id=4882560
I always see this as a quality that I'm missing compared to other coworkers. They'll go for the simple implementation with the complicated usage pattern while I always aim for the complicated implementation with the simple usage pattern (to a point).
We've had a lot of discussion of your simplicity-is-not-simple article which did a great job articulating the trade-offs.
http://raganwald.posterous.com/simplicity-is-not-a-simple-co...
http://steve-yegge.blogspot.ca/2008/06/done-and-gets-things-...
So it's not that your reasoning was wrong. It's that your conclusion was. The right conclusion was, "What the hell were those amateurs thinking? That's idiotic!"
I'm going to go fondle my ALGOL 60 report now...
As I get more experience I appreciate the benefits of languages that try very hard to minimize cases like this, as well as how difficult that is and the trade-offs that must be made.
It's hard to encourage flow type states of high productivity when your mental model of execution either keeps disagreeing with reality or is so full of previously discovered edge cases that it's too complicated to do useful simulations of code in your head.
In fact, even in the example given with respect to Ruby procs and lambdas and returns and whatnot, clearly the author hadn't "gotten things done" in that area (by his own admission, this isn't snark).
So to me, the real point is that thinking you've worked everything out from first principles doesn't mean you actually understand all the implications of the interactions in a complex system. That's the mistake I made as a young man, certainly, and it took me years of learning the hard way that I wasn't always right.
Also: there's always a relevant XKCD. http://xkcd.com/793/
This is why I tried (perhaps poorly) to be specific and talk about smart in the sense of mathematics.
Mathematic smarts isn't always enough to know how something really works. But I still think you were smart enough to realize you might be wrong and to double check how it really works.
Also, you have the GTD mindset to accept how Proc works and try to learn from the experience instead of obsessing about how Proc is broken and ruby sucks.
I used to have this problem. Not the "problem" of assuming you're smart. I mean the problem of assuming that if someone moves the conversation away or disagrees with me, it must be because of something I'd done. Psychologists call this "intrapunitive behavior".
Here's what I would posit happened: you said something intelligent that your companions had no way of countering. Thus, you showed them up and rather than fess up to it, they decided to change the subject.
Humility is necessary to say "yes, I'm smart, but there are the limits of that intelligence." Any smart person can learn enough C++ to get things done if they know Java, but it takes humility to say "I will learn C++ idiomatically and approach it on its own terms, rather than coding up Java paradigms with C++ keywords and libraries".
I suffered at a polyglot consultancy that shifted people around rapidly between stacks and technologies due to overconfidence in this belief; it is very painful for any programmer with a sense of craft and client stewardship. Of course, if all you care about is banging out a piece of garbage that will hold up until the checks clear, then I guess this doesn't matter.
One of the most useful things I've gotten out of my machine learning study and work on MOOCs is second-order learning. I've learned a lot about how I learn, especially under the added and somewhat artificial difficulty of having a full-time job. Very valuable second-order insights.
"Gets things done" is ridiculous. It's a good size-up for very junior level developers, but beyond that... you want "does the right thing". The idea that there's a large number of perfectionists or lazy people who are constitutionally incapable of "delivering" is nonsense. When to stop perfecting and optimizing is one of many important skills, but not a hard one to learn.
So... let's make it, second-order smart and does the right thing.
The unfortunate thing is that corporate american tends to highly undervalue the BS in Physics. While I graduated at the worst time possible time (2008/2009), it took me over a year and a half to find a "real" job. No, I didn't have two + years experience in developing with .Net. I did have 4 years of experience in everything from LabView to Verilog, photomultiplier tube calibration and particle detector design to advanced electrodynamics to CADD and machining, and all the calculus and linear algebra necessary to get there.
As someone who was perpetually frustrated by a lack of consideration when applying to jobs, I'd just like to recommend that anyone reading this comment to always consider a candidate with a background in the hard sciences, even if they don't have the relevant experience.
The net result is that you might end up in a BD-type analyst role, which may not be bad depending on your goals. It wouldn't really hit the engineering target, though.
What will probably happen is that the pool of "data science" jobs will grow, but the bottom half won't be real DS.
I don't mean to imply that a person can become a decent data scientist overnight. It takes years, but it can be done.
On Wall Street this sort of role is called a "quant", although there's less machine learning in finance than one would expect (hard to audit). "Data scientist" is a startup quant. Just as with quant jobs, some firms will only hire PhDs with research experience, while others will take chances on smart people without it.
I'm arguing that it's more serious than this: there are so many PhDs out there right now, companies will have no lack of skilled, well-credentialed candidates to choose from. Going down the "do it yourself" DS route is thus (a) very difficult, and (b) not very likely to pay off.
I ended up switching to CS my last semester, so nobody knows I was a physics major for most of my college career and that basically all of my programming knowledge is self-taught, but I've never felt at a disadvantage to all the folks who bulked up on CS courses in undergrad.
I feel like I have to deal with this all the time -- people who think they're smart and reason their way to tenuous conclusions based upon a ton of seemingly solid beliefs. This rarely ever works, and it surprises me how people who base a significant portion of identity on being "smart" don't grasp that basic point.
There are a lot more confounding variables than there are times when things go perfectly. 6 months of direct knowledge/experience of a particular domain can go a lot further than a year of theory.
In theory, anyway.
The example he gives about FizzBuzz in the Lambda Calculus is perfect. If you're truly working from first principles, then you ought to understand the difference between a theoretical and an acceptable solution.
I'd say that in the context of software, "smart and gets things done" is at a minimum someone with a solid ability to extrapolate from first principles coupled with the ability to problem solve and think critically about what is is they're actually building. Experience helps tons on that front, but it's no better an indicator of future success than grasping first principles.
We have control over the amount of real reasoning we can do about our code. The hassle of the real can be minimized. For example, I like clojure, and there are real benefits to theoretical things like immutability by default, or garbage collection. This ability to reason mathematically while being practical is worth promoting in computer science and engineering. If I couldn't reason this way about programming, I wouldn't be a programmer. Engineering is the intersection from theory to practice, and I appreciate that in computing, incidental complexity can usually be traced back to someone's neglect. Without this, we only have heuristics.
There is no silver bullet, but things can be made measurably better. We can accept reality without giving up on reason. I think to prove it, we can look at the influence of mathematics on real discoveries in history, and maybe ponder if humans are just meant to think this way in order to make progress. I hope my code reflects more order (purity, generality, conciseness) than chaos (special cases, entropy, technical debt).
I would just call the procs thing a 'wart' and avoid using it. I don't think my opinion favors the mathematically inclined programmers over the empiricists.
So, let's think about the kinds of things that get done as a consequence of certain types of thinking.
Tradeoffs.
Now obviously the definition of "smart and gets things done" is open to a huge amount of interpretation, but I think it's clear enough that he means getting things done in terms of shipping useful product. If one does that, then the problems that come with inexperience are self-rectifying.
The problem is if someone is not "smart", they won't learn from their mistakes, and if someone doesn't "get things done" they may lose site of the final goal and fall into mental tarpits as they gain experience.
I have always like the saying "get things done, smart" better which perhaps encompasses this post's thesis better than Joel's original wording.
it's like voyager 1, we thought we knew what the edge of the solar system would be like, but until we actually reached it or at least reached the zone we are in now we didn't know it would behave the way it does...
With code, that is also true, you can't assume it will behave as you'd expect, what if under the given conditions there is a bug in the assembly code generated by gcc when it compiled the ruby interpreter such that the lambda behaved incorrectly? Or even a bit of memory was flipped into an invalid or unexpected state... shit happens which is why we must type, test, type, test, type test... all day to figure out how it really works...