The Rest of the Story (Yammer & Scala)
codahale.com
codahale.com
I like Scala, and I think it's the most powerful and flexible mainstream language available today. Unfortunately, that's the problem.
My own personal viewpoint is that the Nutters and Colebournes are totally on the money. I remember reading a quote from Miguel DeAcaza talking about thread based concurrency programming models in the new N-core paradigm, he said it didn't scale, because "there are not enough PhDs in the world." Despite it's name, I don't think Martin's baby will in the end have lived up to it. Didn't scale, not enough PhDs in the world.
I can vouch for that. Around a year ago I asked him for feedback on Cassandra and his response was both thoughtful and helpful. I appreciated that a lot. Thanks, Coda.
But if everyone thinks this way, those innovations will have one view on the web (where there is an image to care for) and another one of engineering departments, where people get real... and that sucks.
It's not like there isn't already plenty of Scala bashing going around these days anyway.
Was it Language of the Week the week they started building Yammer? Were they trying to follow in Twitter's footsteps? Were they a bunch of Java developers looking for something better than Java? Did someone on the team have previous Scala experience?
Would Yammer be where it is today had they gone with Java (or another language, JVM or not)? Before running into the limitations described in the leaked email, was everyone happy with Scala?
Scala is far from perfect and has some warts, but that's the type of technology that always seems to win. C++ vs Eiffel, Intel vs Motorola.
A lot in the original email is correct (I can't speak for the performance stuff as my skills are more webdev side of Scala), especially the way that libraries/OSS projects/sbt pan out. Something like Dispatch that is a fantastic tool and should be simple is a mess of functions-as-operators and poor to no documentation or examples to the point where for some things even reading the source code is not a lot of help. I'm not going to type out my thoughts on the shambles that was the sbt rewrite at 0.10+... just complete madness.
Not all of HN's discussion is keen on it, but often some of it has good signal.
Scala and Clojure, for their problems (which are not severe, but not to be ignored) have the advantage of being closer to the languages and approaches of the future, in which building comprehensible systems at very high levels of complexity/intellectual intricacy will be even more important relative to performance of executables. I expect that the problems of these languages (including performance) are going to become much less major over time, just as performance gaps between Java and C and between C and hand-written assembly have nearly disappeared.
Spending another 5 years in Java means you're preparing yourself for a 1996 job in 2016.
Why Scala and Clojure? Unfortunately, the 10-year forecasts for Haskell and Ocaml aren't great. I wish they were, because those are great languages. The reason: libraries and tooling matter, and they seem to be mattering more as the years pass (while differences in performance between languages narrow and matter less over time).
Hiring and talent: It's true that you can't hire 50 Scala programmers per month. On the other hand, most startups are far away from being in a position to hire 50 programmers per month at all, much less 50 good ones. Choosing an "esoteric" language works both ways. On one hand, it reduces the total pool of ready developers. On the other hand, I know companies that use Haskell and Ocaml and are able to hire 60-75% of their in-office interviewers while remaining selective; this would be impossible for a Java shop-- there are just too many unqualified Java developers. Choosing a language like Scala or Clojure will do a lot of the filtering for you.
Is the same true of someone using C or C++ today?
C is still the leading language for a lot of purposes: operating systems, runtimes for higher-level languages, device drivers, and applications where tight control of memory use is important. In these domains, C hasn't been supplanted yet-- not even close.
I don't know that C++ and Java have substantial domains in which they're superior to other choices. C++ is a badly-conceived cross between C and high-level languages and Java is the worst of the JVM languages.
That said I wouldn't want to write larger programs in C and would prefer C++ simply because it scales better (assuming the other programmers don't use all the features it has, but confusing code can be written in any language). Also RAII is awesome.
Your statement is also misleading for another reason. Core engine logic might be in C++, but actual "game" logic is increasingly being shoved into scripting languages like Lua.
The other merit of C++ is that it's object-oriented, however awkward its implementation may be (insert Alan Kay quote here). The fact that it's both fast and natively object-oriented is why C++ for the most part replaced C in the game development community, which once had as much C-based inertia as it currently has C++-based.
Is it a beautiful language? Noooooo way. But it has merits.
Edit: before someone responds with "But computers will get faster and make C++ obsolete..." People's expectations for videogame graphics tend to increase as better graphics are made possible. It's a tough fight against an equilibrium, and the fact that the graphics of today will eventually be possible with dynamic languages in the future won't matter for as long as games are unable to reach perfect photorealism -- better-looking future games will come out, and Crysis-level graphics will look dated and shabby by comparison.
For an example of previously-heralded graphics become obsolete even within a single console generation, when the first screenshots of Oblivion came out years ago there were people who truly couldn't believe that they were realtime and in-engine -- but dear God, look at Oblivion now. It's not hideous, yet, but it isn't pretty. And there's no way you could write that in pure Lua today -- not even at ten frames per second.
Where it is true is for languages like Visual Basic, or heaven forbid COBOL. If you are working with COBOL today, or even ten years ago, you're probably maintaining a 1980's mainframe for a 1980's company (VB is probably 90's).
If I were to pull a prediction completely out of my ass, then my complete lack of experience or expertise in the matter says that Java is more likely to end up the VB of tomorrow than the C++ of tomorrow.
While one should keep abreast of current technologies (and languages), there is still plenty of work that requires knowledge of C from the system level to SAAS platform technologies and implementations. It is a 35+ year old language.
If your argument was "learn these knew languages for the advantages the provide given the likely emergence of X, Y or Z", I would buy it. Saying a newer language is a path to the future? Lacking.
http://www.ibm.com/developerworks/library/wa-javascript/inde...
For, myself, I am open to the concepts, but I don't need to be necessarily locked into a given language in this case.
(Also just declaring the semantics of how mutable variables and immutable variables relate in one language turns out to be really tricky problem. Not impossibly tricky; it's been done, after all. But I suspect trying to mix the two in any sort of deep way means that you have unavoidably created a cognitively "large" language.)
If Haskell is going to "win", it'll be on the strength of properties like these, that either you go all in, or you don't get the benefits of them, just the costs. I think instead of 90-10, "90% of the benefits for 10% of the work", it's rather closer to 10-90.
Functional languages will eventually be able to do most parallel computations on vectors and matrices nearly as fast as C. E.g. in Haskell a new project has started for adding support for SIMD intrinsics. Packages such as repa have been shown to be much slower (see the repa papers), where a program using repa with 8 cores (without SIMD) has about the same performance as a single-threaded C program (with SIMD). With SIMD support, Haskell plus repa will probably get a lot closer to the performance of C.
Where functional programming with immutable data structures does help tremendously is in concurrency. Anyone who has written a fairly complex C++ program using concurrency, such as a GUI application, has probably run into complexity issues. There is a lot of mutable state, and no clearly defined model of interaction between threads.
Anyway, in my opinion the real benefit of a pure functional language is that code written in a functional language is often more succinct and easier to reason about. There is no state outside a function (either global or within an object), side-effects are controlled and typing provides neat abstractions (monads, functors, etc.). The primary source of complexity in some functional languages is laziness.
The more I hear this argument (e.g., from the "Python Paradox"), the less I buy it.
Is the theory that true "hackers" care more about the programming language syntax and semantics than whether it has the right libraries for the subject domain of the startup?
Is the theory that true "hackers" care more about whether the startup uses a new and un-tested programming language than traction, likelihood of success, money, or the actual problems being solved?
Do the best "hackers" come in by searching job sites for niche programming languages, rather than referrals?
Even ignoring the huge disadvantages in library support, tooling, community sizes, and so on, are the hiring and talent advantages believable? Are the sort of people who learn Haskell also the sort of people who would be 99th percentile C++ systems engineers for some reason?
An engineer worth hiring is one that is bright, curious, and learns quickly. That said, they should also have an interest in some aspect of what you are producing. I, personally, have searched for jobs on the basis of esoteric / niche items, mainly hoping for something different. The most rewarding experiences have been where I've worked with/brought in good people and built cool things. One of those involved Windows Server and WinCE work, and I personally dislike working on Windows. But the overall project was interesting.
Technologies, languages, approach, and product. A good engineer who is interested will likely have deeper interests and help contribute in one of those four areas.
Finding great programmers who know C++ and Java is not very hard. Finding great programmers who prefer those languages is quite difficult.
This is true, however an inherent danger to Haskell/Lisp/ML-filtering is that this group has also a relatively high percentage of people who think every problem is a nail.
So filter for people who know functional languages, but also have plenty of experience with, say, C or C++ projects.
Also, I think that a fairly large subset of C++ programmers who think that C++ is an ugly and complex language. But it's often practical, e.g. because it is used in legacy software, because it's fast (and provides more abstraction than C), or because many good cross-platform UI libraries use C++.
Many outstanding engineers use C++ (or a particular variant thereof) and like it.
C++ is the only choice to write serious performance critical apps with acceptable abstractions. The STL may be hard to debug, but it's the only way to do certain things.
And now with C++11, we're going to see it starting to give many other languages a run for their money.
The important thing is that people who are using esoteric languages more likely to try new and off the wall things that could lead to a new type of business. A new italian restaurant is not a startup, a restaurant that serves only dehydrated food might be.
If your startup serves italian food then you should probably go hire a bunch of java programmers the python paradox doesn't apply, and arguably most startups by number are serving italian food. (eg. your latest mobilesociallocal coupon startup). If you're truly doing something new then you're probably best off hiring people used to doing new things where there are no established best practices. (eg. trail blazers) You're most likely to find trail blazers on the trail rather than reading best practices docs.
The python paradox is about people not programming languages. (At least that's what I took from it) Also, there's nothing wrong with a new italian restaurant, on average it's probably going to be more successful than an dehydrated food establishment, but if you're looking to create a new nation-wide fad, you'll probably have more luck with dehydrated food than italian restaurants.
If I were to code in Java all day I might as well get a more secure job with a real company.
Of course there is a slighting scale here -- I would still prefer interesting problems, great environments, vesting, etc.
But if I worked at Yammer now I would be pretty annoyed.
As a basically useless anecdote... for random mucking around, I use whatever language sounds interesting. This has been Perl, Python, Forth, CL, Clojure, Ruby, and many others by now. (I used to consider myself a hacker, until I realized I wasn't working on any hard problems.)
But for real business, I'd choose something I know I can deliver with, which is pretty much Perl, C, or PHP for me. I've tried to extend this set by playing in Java and C# a few times, but the interest just isn't there. (Edit: I don't seem to have any problem delivering on boring problems in php, though.)
You'll probably filter out some good candidates, but people who make it through at least stand out from the crowd in some positive way.
http://www.reddit.com/r/haskell/comments/k4lc4/yesod_the_lim...
http://www.reddit.com/r/haskell/comments/mm68o/ghc_74_branch...
In the type of environments where I work, five years experience shipping successful software will advance your career further than five years experience in one particular language.
If you choose a bleeding edge language, what is the risk you will spend time solving edge cases with the language instead of delivering great software? Given everyone's lack of experience in a language, what is the risk you will write something unmaintainable? If your bleeding edge runtime is slower, what is the risk that it will fall apart under load?
It's possible these are acceptable risks for the gains in programmer productivity. That risk to gain ratio would have to be massive to persuade me.
Yes, picking bleeding edge technology will filter out bozos during hiring. You'll also filter out engineers who are more passionate about shipping than new technology. You're left with an high concentration of engineers who have never seen bleeding edge technology blow up in their face.
My other choice is Erlang/OTP, which is 20+ y/o dinosaur, controlled by Ericsson, not a modern language, but very stable and mature.