Will a Two Tier Market For Developers Emerge As a Result of Scala & Clojure? | Hacker News Reader

Will a Two Tier Market For Developers Emerge As a Result of Scala & Clojure?

java.dzone.com

18 pointsjdminhbg23 comments

I think that the distinction isn't so much the language in use as the "quality" of the developer, and I think this distinction already exists - developers who live and breathe this stuff already are way more productive and way more in demand than your average 9-to-5er.

What does interest me is that it seems to me that there is already a two-tier system separated more along the type of work. I've met a few people recently who are consultants in quite esoteric things (compared to "standard") development - often things like IT consulting to companies working on more hard science problems like physics or chemistry/biochem, maybe machine learning these days, and they're frequently using functional or otherwise fringe languages. Similarly, I'm guessing people like Peter Siebel don't have any trouble finding work in lisp. I get the feeling there's sort of a subculture of really interesting work, often in interesting languages, that is just not accessible unless you're exposed to it somehow. Certainly I've never seen job ads for this sort of work.

Any ideas about this? Is anyone out there actually consulting using Scala or Clojure? How did you get into it? How does it work - do you set the terms of the contract, such as which language you use? Any other experiences or opinions?

A lot of my consulting in the last 9 months has been using Clojure and in the last 5 years I have had 1+ years of Common Lisp consulting work.

That said, I don't think that language matters that much - I agree with your first comment. Personally, I don't care much if I develop in Lisp, Ruby, or Java when I help people since most customers request languages appropriate for what they want done.

So when you used Clojure or Common Lisp was that because you got a greenfield project that only you were working on, or were they existing projects that were already using those languages? Again, not wanting to fixate too much on the language but it seems like as a consultant the toolset would normally be dictated by the project, i.e. if there's existing code, what it already uses, or if it's a new project who will be working on/maintaining it.

Could you comment a bit on how you got into working like this? It's always interesting to hear how people get into working in things that are out of the norm.

For the Clojure project, that is what the customer had already started using. For the largest Common Lisp project, they asked me which language I wanted to use; later the project grew to a half dozen Lisp developers; now it is in maintenance mode with 2 or 3 developers.

I wrote two Lisp + AI books for Springer-Verlag a long time ago so I think people associate me with Lisp use.

... is the Java community really so isolated from Ruby, Python, .Net (LINQ), any Lisp, [EDIT: Haskell,] etc? I mean, Scala and Clojure really are nice, but they are hardly revolutionary.

I think the answer is yes.

Why?

Because a typical Java developer was raised to believe that only J2EE programming is real software engineering.

In the words of Alan Kay, "The computer revolution hasn't happened yet." Instead, it is happening. It took two decades for megabit LAN, 300+ DPI printers, WYSIWYG word processors, etc to become mainstream fixtures of the office. It also took awhile for OO to become mainstream.

The revolution comes in multiple, overlapping waves, whose frequency is measured in decades.

Perhaps, but the languages you mention all have clones on the JVM: Groovy/JRuby, Jython, Clojure, etc. Changing languages doesn't mean throwing away your environment and previous creations anymore.

Why, do you think Ruby, Python, .Net (LINQ) and the various Lisps are revolutionary??? Or "more revolutionary" than Scala/Clojure?

Well, maybe, Lisp, 3-4 decades ago, was.

With the exception of Haskell, Scala and Clojure both provide significant innovation over the languages you've mentioned. They are newer languages and they have incorporated many recent lessons from functional programming and object oriented programming research.

They are by no means bad, but what features do they have that justify the multiples in productivity that the article claims? (Genuine question - I'd be happy to be proven wrong!)

From Ruby and Python, the benefits are obvious. These languages are much older designs with less sophisticated runtimes. Their relative stagnation is probably one of the factors that allowed Node.js to explode.

From common lisp, Clojure is a very interesting step. It's much more practical and has a modern structure-reduction style that common lisp predates. It also has all the might of the jvm and that strong and community behind it; the comprehensive std lib that was common lisp's original advantage now looks small and dated in comparison.

As for Haskell, a similar situation exists with Scala. Haskell will continue to be an interesting platform, but the huge amount of work to adopt many of these concepts to the jvm and an object-oriented world have a lot to offer people.

The other hidden benefit is that the jvm can be a new common platform, with java as the new "C glue" layer. You can realistically write a library in Clojure and have it a client to Scala code, which runs critical services inside a JRuby rails app.

So yes, the jvm language community is coming together with big value propositions quickly. They're beginnig to coalesce in a way the CLR family of languages never really has.

P.S., All the languages named here have value and interest and a lot of hard work in them. I have my current set of preferences and opinions on what I'd like to work in, but I don't want to imply any derision towards any of them.

Immutable data structures.

Scala and Clojure both provide higher level facilities for concurrency then Java, Python or Ruby.

Scala and Clojure both have solutions for the expression problem, a very real issue when growing code bases (every OO language should have this)

Clojure provides metadata - a lovely concept of attaching information to an object w/o affecting it's value or needing to 'wrap' it in a class.

All these things add up to a significantly more productive experience. Especially as a project evolves over time.

There already is a 2-tier market. Try being a 9-5er trying to get a job at google/facebook or any of the other big internet companies that only hire top caliber talent.

I imagine[0] that someday most companies will recognize the two tier market formally. Expert programmers would work in Lisp, ML or other Languages of Choice and create DSLs for the 9-5ers to use to write up all the boring details the business needs.

[0] Feel free to replace this with hope if you feel I'm overly optimistic.

You've got it. The specifics tools & languages might be wrong but this is where things are headed.

This is silly. There is no business incentive create a new labor category that can command high rates. Why would they want to make some programmers much harder to replace when they can stick with 9-5ers that are constantly replenished with new batches of college grads annually?

Because it costs less to pay one expert than 30 9-5ers, even if that expert is paid an order of magnitude more. It also provides better results in the form of fewer bugs, better performance and faster changes. Most better tech companies have figured this out already. I expect non-tech companies to figure it out eventually.

It has been my personal experience that companies can't really tell the difference between a good developer for standard applications and a top developer for complex applications. Often, the problem is that the main decision makers don't have a good reference to evaluate developer capabilities. The executives are not technical themselves and they don't have top technical people that can do the evaluation. It is also a problem of who evaluates the evaluator.

This has made me consider a possibility: will this help create a two-tier market for developers?

Already existed for OO. (Back then Smalltalkers were amongst the highest paid programmers.) Now OO is mainstream, so not so much anymore.

Companies are aware of the differences in programmer productivity, but they are painfully unable to recognize, at least at hiring time, who is super productive and who isn't. The best they can do is sort out the reasonably competent from the total clowns. Beyond that... what do they have? Brain teasers? Syntax questions? "Cultural fit"?

If there's going to be two tier market, it's going to be achieved by a revolution in interviewing, not by a revolution in languages.

Reply on news.ycombinator.com