Clojure 1.5 | Hacker News Reader

Clojure 1.5

groups.google.com

252 pointsllambda46 comments

There's a lot of new small improvements in this update which are great for Clojure users, but the fundamental difference between 1.4 and 1.5 is the reducer library which was implemented. It's added mostly for performance reasons: By using this library, you will utilize Fork/Join in Java, but with a functional interface instead of the "gory" one. Very interesting, because the complexity of a typical `(reduce (map ...))` won't change, and you're suddenly utilizing all the cores on your computer. Rich Hickey had a talk about the new Reducers library[1], and I'd recommend you to see it, as it gives you a good understanding of how it works (and why it was implemented that way).

Another slightly interesting thing is the sudden enhancement to read-eval and EDN[2]. That's mainly because of the rough weather Ruby/Rubygems was in with the YAML-exploits, which caused a heated discussion on how the Clojure reader should act by default[3][4].

[1]: http://www.infoq.com/presentations/Clojure-Reducers

[2]: https://github.com/clojure/clojure/blob/master/changes.md#21...

[3]: http://dev.clojure.org/jira/browse/CLJ-1153

[4]: https://groups.google.com/d/topic/clojure-dev/zG90eRnbbJQ/di...

One my favorite features that came out of reducers - short circuiting reduce!

   (reduce 
     (fn [a b] 
       (if (> a 100) (reduced a) (+ a b))) 
     (range 100000))
   => 105

I've often had to use loop/recur instead of reduce because you could not short circuit. No longer.

This is so beautiful, like recur's blessing in disguise.

YES, I was hunting around for this in 1.4, happy to see it's arrived.

cool

Shameless self plug:

If you're interested in keeping up with clojure news, I recently started a clojure newsletter where we cover this sort of thing: http://defnewsletter.com

Here is an idea for you : instead of news ( which is nice but not of urgent concern) , could you send out something like a another list "A drip of Javascript" does( not mine -- I am just a grateful subscriber). Having weekly cool clojure lessons would be nice.

Thanks for mentioning A Drip of JavaScript. I just noticed a bunch of new subscriptions come in at once, and I think that your comment is what triggered them. :-)

This is a great idea. One of the things we want to focus on is cool / useful code projects and how to grok advanced features (monads, ML, async io, etc).

It would probably be better to pitch that our angle is code-heavy rather than news-heavy. Thanks for this feedback. Also, A drip of Javascript is really cool. I just subscribed.

Status Code, Javascript Weekly, and Ruby Weekly have similar layouts. They might be worth checking out for curating-inspiration as well.

Both Javascript weekly and Ruby weekly periodically cover idioms of their respective languages, I like those. Ruby Weekly seems to have more posts about code refactoring and style, which I also appreciate.

There is already newsletter called "Clojure Gazette" http://us4.campaign-archive2.com/home/?u=a33b5228d1b5bf2e0c6... Maybe you two can look what there is common into one news source?

Reducers look interesting! The linked blog post by Rich hickey is an interesting read: http://clojure.com/blog/2012/05/08/reducers-a-library-and-mo...

here is the video of Rich's session on Reducers from EuroClojure 2012 http://vimeo.com/45561411

My apologies for the slightly dumb question, and perhaps this is the wrong place to ask, but:

These sound like a really good thing, but what if I want to use map and not reduce? Is Clojure smart enough to understand that some other function not named reduce should be transformed the same way?

Go ahead and use map (just make sure you use the one from the clojure.core.reducers namespace). The reducers library provides implementations for many functions including map, mapcat, filter, flatten, take-while etc.

It's not intelligently transforming your functions — it's providing alternate functions. The clojure.core.reducers namespace includes reducer-based functions named map, reduce, filter, etc. and you can pick and choose any of them you like. Just list the ones you want in your ns form like you would with any other library.

clojure.core.reducers namespace provides implementation for many functions: map, flatten, filter, etc.

The brilliance of the reducers library is that map is implemented in terms of reduce. Namely, mapping a->b in a collection is the same as reducing the collection (starting with an empty collection) by appending the converted form a->b to the accumulated collection.

How do I upgrade just 'lein repl' to use Clojure 1.5?

    [bin]$ lein version
    Leiningen 2.0.0 on Java 1.7.0_15 Java HotSpot(TM) Client VM
    [bin]$ lein repl
    nREPL server started on port 41384
    REPL-y 0.1.9
    Clojure 1.4.0

I understand that Clojure is just a jar dependency, just need to find where lein's dependencies specified...

For a new project I would have specified Clojure 1.5 in defproject, but for REPL?

This is currently not possible with lein (bug/enhancement), but will be with 2.1.0. See #966[1] for details. As a workaround, any project dependent of Clojure 1.5.0 will utilize that for the repl. So when in a project using 1.5.0, the repl will be 1.5.0 too.

[1]: https://github.com/technomancy/leiningen/issues/966

The conditional threading macros are nice. I've been using them from clojure.core.incubator for a while as -?> and -?>>. Anyone know why the name change?

? in Clojure hints to predicates and boolean contexts while -?> just checks for nil.

Looks like the equivalents of `-?>` and `-?>>` are now `some->` and `some->>`, is that right? Perhaps the name change is meant to avoid confusion now that there are multiple variants, i.e. also `cond->` and `cond->>`, which are different.

It also occurs to me that threading through `some->` and `some->>` is kind of like doing computations in the Maybe monad in Haskell.

I've been using Swiss Arrows. https://github.com/rplevy/swiss-arrows

Perhaps this isn't the best place:

I eventually want to move on beyond ruby and java and try out a functional language. Clojure, Scala and Haskell all seem interesting. Haskell because it's pure functional. Scala and Clojure because they're functional and on the JVM, and out of the two, Clojure.

So it's between clojure and haskell in my mind. Haskell has a great tutorial book/website (http://learnyouahaskell.com/). Is there a great resource like this for clojure? (obviously there are many books, but what's the best, and ideally, does it have a free online version that I can try out before buying)

This is good if you are into roguelikes:

http://stevelosh.com/blog/2012/07/caves-of-clojure-01/

All parts:

http://stevelosh.com/blog/

Thank you!

This looks great, thanks!

Thanks. Just completed peepcode's Clojure screencast based on https://github.com/technomancy/mire

This should be a nice follow-up.

Feel free to reach out to me for more information about Haskell (or Clojure). I think they're both fantastic languages to learn and I have a lot of experience in each.

Will do. Thank you.

I have been going through the O'Reilly book for clojure:

http://www.clojurebook.com/

I like it so far because it relates many new functional programming concepts back to ideas from other languages. In the early portions of the book it frequently shows ruby, java, and python code alongside a line of clojure.

A more interactive way to try out the language would be to try www.4clojure.com or the clojure koans. Both take different approaches to giving you puzzles to solve with clojure.

Thanks!

"Clojure because they're functional and on the JVM"

In addition to that I'd add:

- Clojure because it's a Lisp dialect (and if you've never learned any Lisp, then it's good to add that to your skillset too)

- Clojure because it also targets other runtime, like say browsers JavaScript engine. ClojureScript looks really interesting and I'm sure we'll some very interesting things in ClojureScript very soon

- Clojure because it has STM

Now honestly between Clojure and Haskell I'd say that you should really learn both.

I'm getting quite OK with Clojure. Now I definitely need to find the time to learn Haskell...

Haskell also has STM.

I haven't learned a Lisp. And ClojureScript does look interesting. Thank you.

The trouble with Clojure in my opinion is the slow JVM warmup, especially when writing small command line tools in Clojure, it's a real pain in the ass to run

   lein run myscript.clj

and then sit there waiting for 2 - 3 seconds to see your result. Might sound insignificant but it's noticeable when you're constantly switching between source and result. Obviously the way to get around this is to load everything into a REPL and enjoy the advantages of explorative programming.

This is where Haskell has an advantage, the runghc command is very fast, and compiling your script/program into a static binary means it's quick to execute as well.

Clojure (and Scala) need to sort out the JVM warmup to make them viable general purpose languages.

The bizarre thing is, running Java on the JVM startsup almost instantly.

Yeah, I've been reading about clojure's performance a bit today -- the slow JVM startup is mentioned a bunch. I've been reading that it's not always trivial to get clojure to perform on par with java itself.

From a web application background, are performance gains seen because of clojure's concurrency?

I've seen many, many attempts at "automatic parallelization" over the years. Clojure 1.5's reducers are the first approach I've seen that I think might actually work!

The work-stealing part can be understood as automatic, but Java's FJ is responsible for that AFAICT.

Having to state "do this in parallel" (by using clojure.core.reducers fns instead of their clojure.core counterparts) is not automatic but explicit parallelization.

I believe that the idea of a compiler that will infer the parallelizable parts of your code crosses the limits of computability.

I view the need to use the reducer functions in place of the "vanilla" versions similarly to a compiler flag (you could stick it in the namespace definition and not worry about it for the rest of the code). In my mind, what makes this "automatic" is that in at least 80% of the cases, that's the only thing you would need to do. All of your code using map, reduce, and friends can remain mostly unchanged.

Before reducers, I'd have said that Obj-C's blocks with GCD was the closest thing to "automatic parallelization" (in that you could get concurrent execution without the need to handle explicit concurrency primitives). The problem, of course, is that writing C/Obj-C with closures is a non-trivial shift in design. With Clojure, the way that code is currently written is already well positioned to benefit from reducers.

The functions are in a separate namespace at the moment, but perhaps the ultimate plan is to make them the default? Does anyone know whether that is the plan?

I suppose there should always be an escape hatch to explicitly choose either of the two for a specific piece of code, but the default could be "JVM, please choose what's best for my code".

The reducers versions have the drawback that they aren't lazy because they are defined in terms of reduce; so there is still a place for the core map/filter/etc functions which can operate on, and return lazy sequences without doing any actual work.

Parallelization has some fixed cost, so for small (i.e. most) work loads, using it results in worse performance. So I don't see reducers (or clojure.core/pmap, for that matter) becoming the default.

Generally, the part of a program worth parallelizing is pretty obvious: that that is long-running, must process lots of data, etc. Most calls to map/reduce/filter in the average (Clojure) program are nothing like that.

Finally and AFAICT, ForkJoin can do actually worse than a simple FixedThreadPool (as used by clojure.core/send) for workloads that are "symmetric" and not particularly divisible in subtasks.

I would like to to know if clojure has big memory footprint. Since is doing dynamic class generation and so on. I did find something[1] on stackoverflow but i would like some more info from real production usage.

Thank you all.

[1]http://stackoverflow.com/questions/4058430/how-well-does-clo...

That's it I really need to buckle down and learn clojure. I just got the perfect project today since I have to use a certain Java only library and then do some text processing on the copious results of that library's processing.

Reply on news.ycombinator.com