Shall We Use Clojure?
andrewhjon.es
andrewhjon.es
I'm thinking in particular of this section from O'Reilly's Clojure book: http://books.google.com/books?id=I8KdEKceCHAC&ots=wNiLN6...
They lay out a couple of algorithms in an imperative style, and then gradually morph them into a functional style. It's pretty wild: you go from a story, "do this, then do this, then do this" to a math problem: "your answer is a list composed of f() of g() of h() of another list". (Unfortunately you only get snippets from that link -- it's worth the money to get the whole thing if you're interested.)
Anyway, having followed through their examples and done some of my own Clojure programming, I ended up not so convinced that people who are good at solving story-style imperative problems will necessarily be good at solving math-style functional problems. It felt like a different part of my brain.
This is not at all a reason not to dabble with functional languages, just a potential pitfall to be aware of if you're thinking of bringing a whole team over. And maybe I'm overstating it? I haven't dabbled too much myself.
When you're coming to a new paradigm which is completely different from what you're currently using, then of course it will initially feel hard/different and that you're using a different part of your brain. That doesn't necessarily mean that it's harder for you to program in a functional style, it just means that you're more experienced in imperative programming.
I just have to throw a flag here and point out that Javascript is also a functional language. With Javascript you can declare functions as variables, pass functions as arguments to other functions, return functions as the result of other functions, and use them as part of a data structure. Javascript is much closer to Scheme than it is to a C paradigm.
Javascript is kinda sorta maybe functional if you look at it right, but it's still mostly the same imperative/object-oriented mishmash as Java, Python, or Ruby.
(Java is sort of in between -- it technically has that feature but with verbosity getting seriously in the way of using it.)
I'm weirded out by the idea of writing CPS-transformed code by hand for node.js, but apparently people like it? Perhaps it's something similar.
I like to imagine Node's success comes from bringing some of E's insights to a community just ready to use them, and if the cost is manual CPS, apparently it's not too high. (Relevant E theory at http://erights.org/elib/concurrency/overview.html )
I'm pretty sure you can do all of those things with function pointers in C. void (*func)(void), for example.
Yup, in Perl as well..
object.method(arg) // Java
(.method object arg) ; Clojure
Depends on the audience I'd guess, considering bug-proneness is an almost direct function of LOC (at equivalent skill levels) it can be a pretty interesting note.
Though I'd agree that concurrency prowess should be higher on the list, especially in SOA companies.
When Rails came out, it offers more than just RAD for building web but it also offers integration with the 2 most commonly tasks in building web-application: DB migration and Automated Testing. That strikes to the heart of any type of applications be it enterprise or one-man-show.
Ruby community can't sell Ruby because of its syntax alone by saying that it looks like English (keep in mind that Ruby was invented somewhere in the mid 90's era and didn't exactly known until Rails).
We're seeing the same trend here: promise of concurrency. I also would point out "leiningen" as a build/packaging tools as part of the selling point: there are some portion of Java developers that would swear by Maven (including me, despite its quirkiness/complexity) as the killer app/framework/tool for Java. Leiningen fills that gap for these individuals.
Functional programming in general.
An awful lot of finance (that used to run in Excel) is just side-effect free, compute-centric transformations over data. Time series, for example, have a relatively nice algebra that makes it easy to manipulate in terms of maps, scans and folds. And anything that makes it easier to avoid off-by-one errors is a big win, especially if your traders and quants aren't trained software engineers.
true dat. For example, in Excel, checkout PV ( Click on Formulas->Financial->PV ) Given a $1K@par 10Y bond paying $50 annual coupon at 2% interest, Excel says the Present Value (PV) of the bond must be $1269.48
scala> (0.0 /: (1 to 10).map(x => 50/math.pow(1.02, x)))(_+_) + 1000/math.pow(1.02, 10)
scala> 1269.477
I mean, I loved clojure when I tried it out, I just do not know if I can use it to process data as with other languages with really good libraries (python, for example).
As to processing data like you would with other really good libraries... if you want to do it like that you can. Clojure gives you easy access to Java.
1) What level of support is there for the Clojure toolchain? Is this likely to be around in 5 years from now?
2) We are going to need to hire programmers in the future. Are there people with Clojure skills (or those that we can cross-train) around? In our geographic region? At the price point we can afford? In 5 years from now is it likely that this is going to be a marginalized technology that we have trouble finding skills for?
These are the more important questions, not how succinct the language is. I've had first-hand experience of how technology choices affect companies and it can be very painful down the road.
(Note: this isn't intended as a dig at Clojure, you could replace it with Scala, or any other up-and-coming language choice.)
Just a quick note: from personal experience, I've been a lot more productive in Clojure than in Java, and I have written a lot of Java code. Things like the REPL, STM or FP speed you up a lot, I would say by a factor of at least 2 - 3, maybe more. I have no data to back it up but this is my perception.
So hiring one programmer instead of 3 and paying him 2x more is already good for you.
When I switched from C to Java, I experienced a considerable bump in development speed. Just the collections framework was a godsend, not to mention things like GC etc.
Quite literally I would write things in days instead of weeks. Same thing goes for Java -> Clojure. And just to be clear, it is not my favorite language.
C++ has stl with many of the same collections, and if you want to use auto_ptr, you don't even need to remember to call delete.
I am not saying there are not productivity gains to be had from using java, but as in this case, it typically comes from not having the relevant experience elsewhere.
From a quick glance at the docs glib looks good but it is still too low-level for normal app development (for example, the size of a hashing table created by hcreate() is fixed; so you have to create & copy & destroy a hash table if you want to add more elements than it can hold at a time) and it should be, that's the whole point.
C++ is a whole other story, sure.
My point is - are you sure you just aren't better programmer overall and attributing that (somewhat underservedly because GC and a huge choice of collections) to Java/Clojure/<insert language>?
Well, it's been known for awhile that languages and environments can have a 2X or 3X influence on programmer productivity.
http://www.qsm.com/resources/function-point-languages-table
However, it's also widely held that individual programmer ability and the cultures of certain groups can have even larger affects on productivity.
Switching languages is hard. Managing groups is hard. Dealing with the rest of a large organization is hard. If a group can do fine without switching languages, why would any manager be in a hurry to do it?
Personally speaking, even though I really wanted to love Clojure, I was forced to conclude that Python is more productive.
In my experience, though, metaprogramming is of limited use as anything other than syntactic sugar. And it often leads to very confusing code.
Other language features are far more fundamental and important: libraries, good integration with the shell, coherent error messages (!), readability, tooling, deployment, etc. As a language Clojure is beautiful, but as an ecosystem it has very far to go.
In my experience, metaprogramming is a wonderfully powerful utility for controlling complexity and confusion.
As far as the ecosystem - I used Python for a couple years, it's good. But the Clojure ecosystem is doing pretty darn well itself these days.
Lots of angles to consider not just in terms of monetary. Human's behaviour/attitude/mindset is a very large unknown despite however hard we tried to make a "human" as static as a "resource".
I would rather hire somebody really good for $6k / month than 3 people for $2k each.
But alas, finding the ideal situation is like winning the NBA Championship trophy: you need all the stars to line up including "Lucky" star.
There are lots of good reasons to have multiple developers on a project other than simply throughput.
I've had the same experience. A couple of years ago we decided to implement a rather large part of a project in JRuby (the other part was written in Java), because it seemed to be the right tool for the job: a scripting language that can interoperate with java libraries out of the box.
In retrospect it was a bad decision, because the interpreter error messages were (are?) cryptic (at least compared to what you get with Java) and IDE integration was really, really bad. Cross-language re-factoring was broken and we ended up unit testing almost every line of JRuby code to make sure we didn't introduce regressions when making changes to running systems.
There's a part in Peopleware (http://www.amazon.com/Peopleware-Productive-Projects-Teams-2...) where the authors describe that the choice of programming language has almost no impact on the success of a project. So why add additional risk when there's no proven benefit?
Edit: Also, people always argue that they can get things done much faster in "esoteric" languages, but forget about the time spent on maintenance, bug fixing, explaining code to new colleagues, ...
>Is this likely to be around in 5 years from now?
In 1997 when I graduated, career services told me to erase Java from my resume and replace it with "Borland C++ OWL 2.0" so that I could land a job. (http://en.wikipedia.org/wiki/Object_Windows_Library) At that time, Java = dancing duke on netscape navigator. Right now, I'd give a million dollars to find another OWL 2.0 developer ( ok not a million, maybe $10 )
>5 years from now is it likely that this is going to be a marginalized technology
5 years == light years in tech. Nobody can predict that far out.
>it can be very painful down the road.
If you future-proof yourself by only using the most stable, well supported toolchain, it can be much more painful down the road when you are stuck with some legacy platform nobody is willing to code in because everybody has moved on to the next big thing.
>you could replace it with Scala, or any other up-and-coming language
There should be a statute of limitations on when you can stop using "up-and-coming" w.r.t a language :) Seriously,Scala has been around in some shape/form since 2003. We've had stable releases since 2008. At this point it is no longer "up-and-coming".
I think this common perspective is a little misguided. Programmers retrain fairly easily, inexpensive recent graduates doubly so. With JVM languages, it's particularly true, because the overall environment is so large compared to the language, and all that comes over for free.
I don't really understand the business world's preoccupation with 5 years of experience. My experience has been that some programmers "get it" right out of school, others never get it, some continue to improve into their 60s and beyond. Experience levels correlate strongly with desired starting salary, but not with skill or ability to learn.
> In 5 years from now is it likely that this is going to be a marginalized technology that we have trouble finding skills for?
Lisp was a highly marginalized technology for a few decades there. No doubt you could scour résumés at Monster.com all day and not find anyone who had bothered to put it down. But nobody who actually tried to hire Common Lisp programmers had much difficulty finding them or paying them, because of the prestige. Very few languages seem to enter an actively-hated state (Cobol and Mumps being obvious exceptions); the rest enter disuse and we programmers get all misty-eyed for them.
In short, this is one of those "obvious" business fears that turns out to be completely irrational.
On the flip side, where I work, we use Java 6, JSF 2 and Hibernate: industry standard stuff. Nevertheless, we've been able to fill only one of three open positions in the last three years, after multiple month searches. The size of the labor pool is usually not the limiting factor: in our case, we are in the middle of nowhere and offer below industry standard compensation (though it's very high for the region). Where you are, what you pay, what the benefits are like, what you work on; these kinds of things are going to have a much more dramatic effect on your ability to find developers than simply choosing an off-beat language. Plenty of recent graduates would rather work for less money on interesting problems with desirable tools.
But are you asking the right questions?
Your hypothesis is that anybody suffiently trained is enough and it is easy to scale by having more people.
This work well for contracting companies that effectivelly sell people and not really care about the software. Their model is totally scalable: they revenue is mostly dependant of the number of people billed. And too much productivy is more a nuisance. They just need not to be worse than other contracting companies. So do the same as everybody else, be sure to have low operationnal costs and you are fine.
But from the software side, this is the opposite. Software don't scale. The bigger the code base, the more it cost to maintain. Too many people working on it and too much turnover and your team start to spend more time in meetings, politics or helping newcomers than getting things done.
As a manager, while this is far more satisfactory to manage bigger teams, your role is to empower your team so it can do more with less. And any tool that can give you an edge is a huge asset. If a better programming language mean more productivity, use it. If a better SCM can help, use it! If faster computer ensure that your devs stay in the zone, buy them.
That the point here. You can do more with a good team of 10 people than with an average team of 50 people (where 10 peoples will be consumed just to manage others).
For the record, I am not implying that Perl is above criticism. But, showing a single arbitrary perl code segment, and touting that Clojure doesn't look like it is not fair comparison.
I don't think he was trying to claim that was idiomatic perl. Rather he was saying that clojure is succinct and still very readable.
I have not written a single line of clojure, and don't know how idiomatic clojure looks like. I tried to translate his example to clojure using my very limited knowledge and looking at the examples on ClojureDocs site:
(sort #(compare (second (re-matches #".*([aeiou]).*" %1)) (second (re-matches #".*([aeiou]).*" %2))) %)
The funny thing is, when I just add spaces to the Perl code segment, it looks quite readable to me. But then, I probably know more Perl than him.
sort { ($a =~ /([aeoui])/)[0] cmp ($b =~ /([aeiou])/)[0] } @_And it's a valid argument. There's sometimes a tradeoff between terseness and readability. Of course, readability suffers if the code gets too verbose as well.. go figure.
"Perl is a language where you can choose to be so terse as to be nearly unreadable"
You can actually choose a "level" of succinctness that fits the task at hand and your personal style. This is a very cool feature of Perl that it should be praised for. I agree with GP that the OP intentionally misrepresents it.
The Joy of Clojure goes into to Clojure philosophy and explains the "Why?" of Clojure. Clojure Programming is divided into five parts, and parts III and IV provide a good overview of how to set up your environment and structure projects.
But the best book for learning the language is Programming Clojure (http://www.amazon.com/Programming-Clojure-Stuart-Halloway/dp...) by Stu Halloway. Stu works alongside Rich and understands Clojure at a deep level, but he's still in tune to the beginner's mind and is able to clearly explain concepts and provide the context you need for the ideas to resonate.
Joy is the one book I consider a necessity to getting a deep understanding of the language without spending as much time as you might attempting the same understanding on your own.
I really like some of these languages. Clojure is something I've worked with but haven't built a complete product from, but I'd like to make something standalone.
I just wish there was a single website devoted to re-creating the terribly boring "Employee Directory Website" app in each language of choice.
It would be even better if the creators of said language would go through the time to make the example site so I could see how they imagine a CRUD layer working, or sessions, or REST/XML data, or the sql connection.
For every "new" language it seems I can find a hundred examples of fibonacci sequencing (or more recently everyone seems to enjoy creating 'twitter in 5 lines of code!') but almost no real "boring web stack" examples.
The best way to gain traction and to get others to use your favorite tools is to write code and implement projects on time and under budget that are reliable and easy to maintain. If you do that, then the tools that you use will be in demand. Others will want to use them and they will speak for themselves.
Aha! I just had a really good idea! Let's call it "Clojure folding". It's like code-folding only instead of hiding lines, you pull up all the code into one line. Some of the internals may be elided - and revealed with a tooltip.
I also like the idea of coloring the entire content of the parens, so that you can embed a large item and then clearly see that you are now back in the containing list. Indentation doesn't do a good enough job, nor do rainbow parens. Further, you'd get a good intuitive sense of place in the code if you have a consistent series of colors. Emotionally, you'd know that if you're typing read you're like 6 layers deep - and that's really deep, in the danger zone, and you probably want to pull some stuff out into definitions.
Chris Granger, feel free to include these ideas in Light Table.
In fact, now that I think about it, I don't think there's any language I'd prefer to read over Clojure.
Nice article, BTW. Good intro for those not familiar with Clojure yet.
Likewise, if I find a more comprehensive tutorial, will post as well.
The VM is optimized for long-running applications, not brief scripts............ bang those out in whatever.
For interests sake, I typed "java -version" on my new MacBook Pro now and it took 0.9 seconds. Second time it took 0.11 seconds. It's still unacceptably slow, in my opinion.
(PS: should mention that it's fine if you don't anticipate editing said scripts often)
But as a high-level language for high-level tasks, for gluing pieces of code and calling libraries all around and for dynamic chancing stuff like web programming, Clojure really shines.
If you're worried about clojure not being fast enough where java is, you've always got the option of prototyping in clojure then dropping to java via clojure's interop where you have to.
A bigger drawback to Clojure, in my experience, is that error messages are complete shit (most errors will yield stack traces to compiled bytecode, so you get java traces with a layer of obfuscation in that they're tracing the bytecode generated by clojure). The same issue exists when trying to improve performances, you get performance counters for java bytecode and have to match them to clojure code "by hand".
So, in theory you can write Java-in-Clojure and see similar performance between the two, as the just-in-time compiler will not be doing any more work for Clojure than for Java code. In practice, the code will not be as attractive as plain Clojure.
If you want to learn functional programming _for its own sake_, learn Scheme (Little Schemer, SICP, Seasoned Schemer) , ML (either OCaml or SML would do), and optionally Haskell.
Common Lisp is worth learning, but majority of Common Lisp code is not very functional: idiomatic Common Lisp code readily takes advantage of cons cell mutability and uses do/loop macros; macros and CLOS are the salient feature of Common Lisp that are worth knowing about.
(I intend to return to Clojure/Python on the next project because Java really makes CRUD style web programming a PITA).