No Silver Programmers
knowing.net
knowing.net
Look at the graph referenced - it describes, on a scale of hundreds of minutes, how long it took programmers to implement something to spec.
My biggest problem with this is the time scale. More advanced programmers build generalized solutions to problems and build on previous abstractions. There's no clear limit to how clever someone can be while building abstractions, so the gap between programmers will become greater over larger time periods.
Taking a sample of developers creating small, custom websites, better developers will have factored out a lot of functionality into a general purpose web development kit with metaprogramming by site #3 (i.e. created a Django, Rails).
Is it unreasonable to accept that:
1) Creating a site with a decent web tool kit is 20x faster than rolling your own with mod_python, and
2) Metaprogramming for creating an (elegant) web tool kit is beyond the abilities of a section of programmers?
Now, the barrier to picking up Django or Rails is much lower than writing one, so perhaps this advantage is somewhat less important in a world with open source and extensive libraries?
Well, I would argue that in a way, the efforts of the creators of Rails and Django have been multiplied so many times by all the users that they, in effect, have increased global productivity by a factor so large it dwarfs the productivity of a single programmer - so their 'productivity', as measured by the impact they have on the world, is a hell of a lot more than 20x the impact of Mr Blub programmer in the cubicle next door.
Why stop at frameworks? What about languages? What is Guido van Rossum's impact on global value delivered by programmers? What about the developers of the microchip? I'm pretty sure my personal productivity would be lower without that. The invention of the internet, which allowed for free sharing of knowledge? What would the productivity of a programmer who invents general purpose AI be? It really depends on how philosophical you want to get.
So, sure, the difference between developers creating an application to spec on the time scale of hundreds of minutes is unlikely to be 20x. But that's a garbage metric, and as always, GIGO.
Rails is three-and-a-half years old, as a public project. The alternative frameworks are about the same age. There are programmers with ten years of PHP experience; there is nobody on Earth with so much as three years of Django experience. Give the newbs some time to settle in before you declare PHP's long-term victory. PHP has had a long headstart.
To have a successful startup that makes a 2007 list, you have to start learning programming before 2007. Back in 2003, PHP was pretty much the name of the game, unless you enjoyed being married to Microsoft or wanted to construct vast mountains of overdesigned, stateful, undeployable Java middleware. Or unless you were writing your own framework from scratch in an alternative language, like DHH, or the Django folks, or the Redditers.
Careful about saying 'least abstracting' though - if it really is faster (not my experience, but assuming it is), then although it might not appear to be a powerful abstraction, it actually is, at least in some way.
Your abstractions (done well) are exactly what I'm talking about - long term productivity. Given 100 minutes you may well be well behind (if you've even started actually coding), but will that still be true 100 days later if the other guy is writing it in straight, hackish PHP?
There isn't much data on single average programmers taking years to do projects because they just don't work alone and on one project for that long. A mean measure probably isn't a good way to break down the difference as it's likely to be infinite far too often.
That seems right to me, it seems like only really Hard problems allow people to show what they are made of, and the easier problems keep the pack more bunched together.
It's an important distinction, because a lot of programming is not of the demanding sort described above. It might even be that for some jobs, in some job markets, you don't want to try to hire the best. The cost in salary, recruitment, lost time and turn over might trump the 2-3x gain a top level programmer would offer in a non-challenging environment.
There are tough problems in programming, but unless you are breaking ground in p-np complete type proofs, are there really any day to day problems in almost any development setting that a decent programmer can't attack?
Often, five man development teams create products that are very successful while thousand man development teams create a product thats useless and basically has no value. Thus the five man teams were way more productive. This happens all the time.
Also, the same principal can be seen on an individual level within projects.
There ya go pal. It would be a waste to do a peer reviewed study just to confirm something so obvious. You could even say that they reviewers would be "less productive" if they spent their time doing so;)
Like comparing automotive designers and mechanics.
Coder is just too big of a blanket. Articles like this are irrelevant.
That kind difference is only possible in languages that are more expressive & condensed, where you spend most of your time thinking not typing.
Actually I recently thought about the "dependency on IDE makes you miss out on cool languages argument", and you could basically just turn it on it's head: rejecting IDEs makes you miss out on a lot of cool features, too.
Another interesting statistics that has been researched throughly is that productivity in LOC/hour is pretty much constant for programmers in all languages, which pretty much translates into higher productivity with more condensed languages.
Check out these links for some real data: http://norvig.com/java-lisp.html http://wwwipd.ira.uka.de/~prechelt/Biblio/jccpprtTR.pdf http://www.flownet.com/gat/papers/lisp-java.pdf
And this research was on a toy problem, not a real world project, on those the differences greatly intensify.
Also, the condensed style of scripting languages might waste more brain cycles than more verbose code when trying to understand it? I enjoy the condensed style and clever things you can do with it. But clever things have a tendency to also require cleverness when trying to understand what they do.
The reference was somewhere in the PDFs I linked to, and its not that far from the truth from my experience.
Boilerplate is a sickness. Even if your IDE is generating it for you, you still have to read, interface, and interact with it in your daily operations. Any higher level abstraction which allows you to remove boilerplate and redundancy leads to greater robustness and flexibility in your software.
I'm not saying that playing golf with your code and turning it into line noise is a solution. That merely results in higher level of complexity rather than a higher level of abstraction. It becomes brittle. But when you can apply more powerful concepts to the solution of a problem, fundamentally complex operations can be expressed in a simple manner.
New abstractions definitely have a learning curve. I believe that this is a separate effect from mere cleverness or an obfuscating complexity. The advantage of this learning curve is that once the concepts are understood, future effort is eased and the new principles become natural expressions of solutions. A professor of mine once told me that Mathematics is driven by the creation of new, more powerful notations. Being able to describe an idea in a more succinct way leads to the ability to reason about a problem space and discover the underlying fundamental principles of its operation.
It may well be the case that it takes a more dedicated, talented, or skillful person to write code at higher levels of abstraction. The innate curiosity and insight needed to pursue the greater goals of software excellence can be distressingly uncommon. I believe that the balance of the issue is offset by the force-multiplying effect of powerful languages, designs, and abstractions as well as the social effects of being able to accomplish more with fewer people. Reducing the social and management overhead of a project results in greater individual productivity, improved group cohesion, and more effective communication.
Any approach which can eliminate the extraneous details from the expression of the solution to a problem and which allows us to work with the raw core of its nature enables us as computer scientists and software engineers to deliver more robust, effective, and elegant software solutions.
And you might even find yourself to be happier while doing it.
Maybe the "Boilerplate" aspect of Java is a bit overblown. There are interfaces, but I think they are a necessity because of static typing (how do Scala and Haskell deal with it?).
So it is really the pros and cons of static and dynamic typing that are to be discussed. There are boilerplate things like the awful EJB stubs, but everybody hated them from the beginning, and I for one managed to avoid using them for all those years. They are not Java, just something stupid somebody implemented with Java. You could also create a horrible framework with lots of boilerplate code for Ruby, after all.
What else? OK, the new generics are too verbose, I hate them, too. And then there are the curly braces and mandatory brackets for function calls, but I think that can't be what everybody hates (Ruby is more verbose with do end instead of brackets, isn't it?).
Really productive programmers win by writing themselves APIs as they go. These can be reused in the future (hence the productivity multiple grows over time). As PG and others have said multiple times, you can be way more productive if you design a language for the problem domain, then use that language to solve the problem. That is what more powerful languages allow you to do.
It's not so much about trying to condense lines of code, it's about how many things you have to focus on at once (the fewer things you need to pay attention to, the easier it is to work with). Since any modern language handles all the minutiae of disk access for me, I can read and write to files with one or two lines. I don't have to care how it's done.
If you think I'm just stating the obvious, I rewrote a project in my last job in about three weeks that took the original team of four people two months to implement. The only thing that allowed me to achieve that kind of speedup was that I used an existing SOAP library (this was a web app with a web service back end for the data) instead of all the kludgy custom XML parsing they were doing. They had different code on each page of the web app for handling the webservice calls. It may sound retarded (because it is), but I have seen that kind of situation a hundred times, so I have to believe that that is the kind of code the average programmer churns out.
I don't claim to be a rockstar either. I know I've written a ton of horrendous, kludgy, redundant crap in my lifetime too. I think it just happens when you don't have a clear idea of what the end product will be, and don't take the time to refactor as you go.