Life of an Implementer: If you're good at it, don't do it for someone else
undev.wordpress.com
undev.wordpress.com
In reality, this simply isn't true. Case in hand: I like distributed databases. However, I alone, have absolutely no use for one. Nor can I (simultaneously) work on every single aspect of a distributed database: data replication and MVCC, network layer, B+Tree implementation, monitoring, configuration, bindings to other programming languages.
When I program on my own (and every programmer should) and for myself, I typically write much simpler programs: tools and utilities (and that's typically the products that "micro-ISVs" sell), conceptual programmings (e.g., I am curious about a certain algorithm, so I'll go and implement a toy version of it) or programming for fun or mind expansion (e.g., re-inventing the wheel, but using OCaml and Lwt), or for career development (e.g., practicing C++ on my own as I primarily program in Java at work).
Only a limited kind of programming that I do on my own (concept programming, mind expansion or "career development") is as technically fulfilling as the programming I do at work. There's no way I could do this kind of programming for a living: I am not delivering a usable product. Paxos algorithm is the prime example of it: I can implement a toy version in a weekend, but it would need months of further development and testing and years in production before being mature (and even then, I guarantee, there will be bugs in it). It would need a real world use case for the algorithm (that I, as an individual don't have), it would need operations engineers to deploy it to production, it would need other programmers to build systems on top of.
The other approach is to develop products that I, as an individual, can build on my own. There's wide variety of this kind of software: project management tools, productivity tools, business applications. There's plenty of people who want to use these kind of systems and there are many micro-ISVs who continually build this. However, while this might (or might not) pay better, it simply won't be (at least for me) challenging enough to be interesting. There are some for who are able to find a niche where both "sufficient challenge" and "ability to do it myself" intersect, but I am not one of them. Not to mention, I would be spending at least 1/2th of my time doing book-keeping and accounting, technical support, marketing and the like. No thanks.
Finally, I could take an "easy" job doing something other than programming that gives me time to do work in the evening. Problem with that approach is, again, it's going yield LOWER happiness than what I have: I am spending a huge chunk of my waking hours doing something I don't enjoy, then when I go home I do something I enjoy, but which (almost never) sees the light of day. In addition in order to spend my time doing something that is a "meaningful leisure", I have to give up time that I could be spending traveling, working out, pursuing other "meaning leisures" (reading, photography) or being with my girlfriend, parents or friends. That sounds like a surefire recipe to develop at least a mild kind of depression. Indeed, individuals who have a deep passion to do X (but where X has a manageable career route), but are unable to find a job doing X, are some of the least happy people I've known (janitors who spend or nightclub bouncers who their time doing physics and mathematics makes for feature films or chapters in Malcolm Gladwell books, but it's a painful way to live).
Let's consider the more conventional role: working for a company (big one or a startup) as a programmer. In this case, I spend a much larger chunk of my time programming (yes, I also spend time in meetings, spend time dealing with operational issues -- but that chunk would be much larger if I were "my own" software company); after this I go home and can choose to either program or go out and do something else (depending on my mood). This way I am spending most of my day doing activities that I find to be a meaningful leisure.
Suppose I do start my own company: I can go two ways. I can go the "micro-ISV" route and work for myself, or I can start a company that's expected to grow. Again, I'd much rather choose the second route: it's much harder, much more prone to failure, but I'll be able to build something that will scale sufficiently to allow me to focus on elements that I find more rewarding (while employees I would hire are able to focus on elements they find more rewarding).
Edit: just as a clarification, I am not saying that all "micro-ISV" work is boring. I just haven't found anything that I could (reasonably) build into a product, such that working on that problem would be (to me) more rewarding than working for somebody else. This could still exist, but it's much harder to find.
People always start out at implementors (or at least they should in my opinion) and as they implement they develop a 'style' and an aesthetic for how things should be done. Sometimes that comes from learning the hard way, sometimes it come from re-implementing or re-working the same code for several different uses, sometimes from reading an insightful paper on a similar problem. After a while, they will have more ideas than they can meaningfully implement in a lifetime, but with the help of other implementors they can make wonderful things.
So they begin teaming up with other implementors, some not as experienced as they and perhaps others with different histories. The build bigger things faster, and begin to appreciate that working in parallel is faster but not more efficient if at integration time everyone has to rewrite what they wrote. So they start spending time thinking about how the pieces of this bigger project should fit together, they keep this complex inter-relationship in their head as they work with various team members who are working on other bits. At some point the part of the implementation they own is falling behind because they are constantly on the move, talking, listening, advising, questioning, deciding, on various bits about the project. As they weave and integrate more and more bits they begin to discover the subtleties of a thousand different things about how stuff is put together and recognize the benefit of finding people for whom that particular part of the implementation space has fascinated them for years and they know inside and out.
They will find themselves giving advice on how something should be done to an implementer who is doing the work and wondering why it should be done that way when they could do it more easily some other way. They may find themselves recounting trying it different ways, the advantages and disadvantages of each, and the outside influences that perhaps this particular implementer isn't aware of which have guided the opinion of how it should be done. They may find that the implementer dismisses them as 'out of touch' because they aren't doing the implementation themselves. They might chuckle at that, or if they have had a long day and they aren't looking forward to the unsatisfying 'when you've done this longer you'll understand' conversation they might be a bit short with them.
Don't worry about what other people do or how they're rewarded for it. Focus on doing what you do as well as you can do it, and seizing the opportunities where it'll have the most impact, and the rest will come naturally.
Somewhere above you in the chain of command you need to have someone with the desire and ability to Do Things The Right Way and the political capital within the organization to make sure it happens. Otherwise you're at the mercy of people who don't give a shit about software development (why should they?). As a software developer, that's not a position you ever want to be in.
Of course, even in big software companies this often happens. But at least you'll have a chance.
#1. A great developer is worth much more than
they gets paid in an organization
#2. A great developer could create much more value
for themselves if they worked for themselves
#3. If you can't go work for yourself now, take
a non-developer position to leave yourself
energy at night for developing your own project
#2 is more of a stretch - you need other skills to start your own business and capitalize on your strengths (edit: the reason many of us follow this site).
#3 is an interesting idea that I haven't seen expressed elsewhere.
It's actually a pretty old idea. The world is full of aspiring actors waiting tables, aspiring writers working at Starbucks, etc.
>> full of aspiring actors waiting tables, aspiring writers
(not arguing the same point myself - if you want to become a great writer, you should be writing all the time. This might make sense if you have already developed your writing skills sufficiently).
I find that I still have energy to think about programming when I get home from work - that's why I'm on Hacker News or Proggit. I just don't have energy to actually program. Those are two different skills, and it seems actually programming draws from the same pool of energy as keeping customer's orders straight, not from the pool of talking about programming.
It can be done, but it's very stressful, both because you're pretty much spending all your waking hours either working or "at work", plus you usually have actual responsibilities at your day job that you're trying to get done while already tired.
I think that a lot of people confuse "value to the firm" with "intrinsic value"
Of course, in most cases management has the exact opposite view -- implementation is an unfortunate cost of doing business, maybe we can get some people in India or Russia to do it on the cheap, while the important functions of business (marketing and sales) are done here.
IMO, if you call yourself an "implementor" then you have already lost the battle against being commodified.
Anyone who can (and wants to) do it on their own, will. In their good time. There's no point in telling them to do it.
Steve Jobs is not an implementer. Wozniak was one of the most brilliant. With the collective talent at Apple, surely they could stop "working for the man" and live their dreams.
Bad or slow implementation can kill the best of ideas but a great implementation of a mediocre product will go nowhere.
It's more about assessing your true strengths and weaknesses, finding a team where your talents are a good fit but get a stretched enough that you're always growing at a healthy pace.
If you feel under-rated maybe it's more about your communication than being under-appreciated as an implementer.
Needing medical insurance will change most peoples' perception of this argument (assuming they live in the US).
Is everyone frothing to themselves at their desks, muttering that they could do better on their own and should quit to start their own business?
Programmers have some of the biggest egos I've ever seen out of any profession. We give Wall Street bankers a run for their money when it comes to how self-absorbed and how self-aggrandizing we are.
So very sick and tired of coders belittling the role of everyone else in this society - 95% of coders out there wouldn't even remotely be able to cut it on their own without the umbilical of a stable, sizable employer. There are a lot more skills one needs to survive in their own business than simply the ability to implement software.
I switched to software development in my 30s, after a career in PR and marketing. Talented software developers ARE some of the most impressive professionals I've ever met. And frankly, many many many marketeer and PR people deserve the ridicule. Heck, maybe even 95% :)
I the case of one talented developer, I was tempted to ask them the following question but held back: What makes you think you could do this kind of work at a high level as an entrepreneur when you are struggling doing it for someone else?
Of course, there is also the implementers that want to do nothing but implement and would not want to play political games even if they had the time.