ClojureScript 1.7: ClojureScript can compile itself | Hacker News Reader

ClojureScript 1.7: ClojureScript can compile itself

swannodette.github.io

536 pointsprogramnature106 comments

> We grabbed a string out of CodeMirror, read it via tools.reader into persistent data structures, passed it into the ClojureScript analyzer, constructed an immutable AST, passed that AST to the compiler, and generated JavaScript source with inline source maps and eval'ed the result. All of this happened inside of your web browser.

This is just fantastic! As David himself mentions - this might not be extremely pragmatic but it sure as hell as cool. For me it means that I can get back to writing my web-based Clojure book without being dependent on Himera[0], which although great, is really hard to work with.

Awesome work by @swannodette and team!

[0] - https://himera.herokuapp.com/index.html

> without being dependent on Himera

It horrifies me that anyone would have been dependent on Himera to do real work. While I'm proud that it filled a gap for a few years and consider it a successful experiment, I'm happy to know that it's effectively unnecessary now.

> horrifies me that anyone would have been dependent on Himera to do real work.

Haha! Well, it was not real work - just a fun experiment[0]. Thanks for Himera, though. It was great while it lasted :)

[0] - https://github.com/prakhar1989/gitbook-plugin-clojurescript

Is your book up anywhere? Would be awesome to see it in action. Even if not, can you plug your book a bit?

I don't pretend to know front end dev too much; I don't program for a living, but it seems the 'bazaar' model of layering, patching and piling things up is making that world unnecessarily complex. If I understand this thread correctly, you lose Google's Closure optimizations(?), you go round about AST->compiler->JavaScript and you can eval the result. Is the eval equivalent to Scheme or Common Lisp? What are its limitations? I like what I see of ClojureScript as a language syntax and some of the underlying concepts. I am playing with PicoLisp and PilOS (PicoLisp running on a VM - not really a Lisp Machine), and the simplicity and directness of it is refreshing. The layer of abstractions are minimal. I guess that's the same reason I like the J programming language. I am astounded by the amount of work and the accomplishment here, but I have to wonder if we can't see the forest for the trees any longer in computer science. Currently reading 'The Architecture of Symbolic Computers', studying Shen, PicoLisp, Racket and the J programming language. Perhaps my naivete of not having to work in this field, and the luxury of being able to dabble across languages and systems has shifted my perspective?

EDIT: How do you debug your work? What traces do you get back? Can you still do REPL (like Emacs/Slime) this way?

One thing that's great about Clojure and Clojurescript is that they run on Java and Javascript platforms respectively, and are easy to interoperate with those platforms. For a business that wants to crank out some code, being able to use the libraries that others have created for those platforms makes the decision to use them for production systems much easier.

ClojureScript is probably not the best language to use if you want few layers of abstraction. I personally love it, though.

The best way to think about this development is that it is a client-side compiler for ClojureScript. In client-side development, you are constrained by the limitation that all code must be in JavaScript to be executed by the Web browser, which is why this compiler "compiles to" JavaScript instead of machine code. JavaScript is the assembly of the Web, right now (until things like WebAssembly get more useful and popular). Once the ClojureScript is compiled to JavaScript, the JS can be executed by the browser (which of course involves an additional JIT compilation step). This might sound "dirty" if you are used to thinking about non-Javascript development, because it involves two separate compilation steps, but it is unavoidable, once again until something like WebAssembly becomes popular.

So, this is a client-side compiler for ClojureScript. A server-side compiler already exists, which is what you would use if you wanted Emacs/Slime support, or if you were developing a "real" Web application for deployment, because in that case you would want to pre-compile to JavaScript on the server, and distribute the compiled JS to the client.

It doesn't really make sense to talk about Emacs/Slime running on the client, until someone makes Web app for it. You could use a socket-based REPL to communicate from the client to the server but I don't really see the benefit, when you could just use the server-side compilation model and avoid the network overhead.

ClojureScript isn't particularly different to how most other languages work. You have a parser (analyzer) that turns text into an AST, then a compiler that turns the AST into the target code. In this case it's Javascript, but the same principle applies if the target is assembly or bytecode.

ClojureScript is now self-hosting, which means that you can eval any valid ClojureScript, so yes, eval in ClojureScript is now equivalent to the eval in languages like Clojure, Racket or any other Lisp.

Usually when a language is bootstrapped, that becomes the new norm. For instance, Go was originally mostly written in C, but since 1.5 Go has been written in Go. However, the Java version of the ClojureScript compiler can leverage Google's Closure Tools, which include a rather useful Javascript optimizer. This means that we'll still want to use the Java version of ClojureScript in most cases.

I swear David Nolan does not sleep.

This is one of those things that people imply when they talk about "rockstar developers". Those who get so much stuff (of relatively significant achievement) done and have time to blog about it, and have time to speak publicly about it, and have time to further engage the community around them.

...and eat and sleep, of course.

And most of the time also work at a day job! And take care of their kids! I'm simply amazed.

You can do that for about 25 years or so and then you're going to slow down no matter what.

I do in fact enjoy sleeping immensely. As alluded to in the post while I tend to do a lot of "driving" - this release was the result of a broader effort by the growing Clojure & ClojureScript communities.

And we are very thankful for all your contributions David.

Thank you so much.

Take it as a compliment man. I just think you do great work. Hopefully I'll see you at the bar again at the next conj.

All difficult problems in the Clojure community are resolved by sleeping, preferably in a comfortable hammock.

Let's try not to judge people based on their hair cut.

The most interesting case for me will be fast starting clojurescripts as the author points out.

The one area I have not been happy with is clojure's startup times so hopefully this fixes that going forward.

It can be very fast to get things running. Here is an example using "desktop" JavaScriptCore on OS X: https://twitter.com/mfikes/status/626884586392453120

This also means that we can use clojurescript as a scripting language. Will make some systems automation much more fun!

Sure. There are a lot of scenarios where the 300k overhead hit isn't going to matter all that much.

Not that 300K should be considered a huge amount by today's standards anyway.

Well, that's 300k of gzipped code, so while the bandwidth isn't huge (300k is barely a large image these days), that's a ton of code to be parsed, and might be too much of a CPU hit for anything mobile.

It is actually pretty good for mobile and startup time isn't an issue. We have variants that run on iOS https://itunes.apple.com/us/app/replete/id1013465639?mt=8 and Android is in the works http://tahmid.me/posts/2015-07-15-bootstrapped-cljs-repl-for...

Oh, my comment wasn't at all clear, but I was talking about inclusion on web pages, where you'd be flicking forward and back between them. In which case even 50ms on top of the rest of a page load would be hurting you.

I'm sure it's fine for a SPA, and for a mobile app it's a total non issue. Actually I'm planning to use it for one with React Native myself.

Luckily, people will probably not be including the full compiler lightly; I just wanted to point out that it's not as cheap as a jpg of the same size would be, an impression I thought the parent comment was giving.

You'd probably not want to use this on websites, unless you want to be able to compile code directly in the browser, such as when you're making an interactive tutorial or similar. Otherwise, you're better off with stripping out all of those parts.

> Oh, my comment wasn't at all clear, but I was talking about inclusion on web pages, where you'd be flicking forward and back between them

On the other hand - in case of single page webapps it would only be loaded once and could be made to load only once thanks to browser caching.

It would load only once SPA or not, that's the point of browser caching.

He's referring to parsing the JavaScript on page load.

Ah, missed that bit ;-)

Also, check out http://clojurescript.net which is running bootstrapped ClojureScript. It starts up instantly on an iPhone.

There are a lot of commonly used mobile libraries that are larger than this. Some of them (e.g., JQuery Mobile, which weighs in at several megs) make this look tiny by comparison.

The equivalent jquery mobile size - minified then gzipped - is 53kb.

I'm only vaguely familiar with clojure, but why would you want to use clojurescript over clojure?

For what GP mentioned ("systems automation") depending on the JVM (as one would with Clojure) is kind of a non-starter, due to the long startup time. By contrast, Clojurescript (essentially the same language, but compiled to JS instead of JVM bytecode) can run in something like Node.js, where startup times are considerably faster.

Edit: this new development (CLJS compiled with CLJS) is remarkable because previously, compiling CLJS required a running JVM. One could still compile automation scripts and run those, but that wouldn't really be true to the "scripting" style anymore.

You can dynamically compile and run ClojureScript scripts so long as you have properly hooked things up to a JavaScript engine. I've started on an effort that makes this easy OS X, wrapping JavaScriptCore and I could see similar efforts for Node.js. Here's what I've been working on: http://planck.fikesfarm.com It can run scripts you have on disk, etc, and it has basic file I/O capabilities at the moment so you can write simple text processing scripts, for example.

planck is a really exciting project. What IO capabilities do you think are missing at this point?

Just for DX / ergonomics, I'd like to mimic more of the file I/O stuff that you have in Clojure, especially the ability to cope with streams. And then perhaps having network I/O would be interesting as well.

Then there is the `clojure.java.shell` namespace that could prove very useful for scripting.

It actually isn't the JVM startup time that is the problem. The JVM starts up pretty quickly these days.

Clojure itself is the bottleneck. Which means it is at least fixable... but seems to require a huge effort to fix.

Since none of your other responders said this, I will also point out the most common/obvious reason: you're building a website in Clojure, and want to continue using it for your front-end work, too.

"Because Clojure rocks, and JavaScript reaches"

-- Rich Hickey

(or so I've heard)

I have been looking at functional languages for frontend. But, I guess I am one of those people who just can't unsee the parentheses in lisp languages .

How does clojurescript compare to elm/purescript ?

You have to use it for some time, and then the parenthesis will just disappear.

This is really true. I've recently moved to CLJS from Ruby/vanilla JS, and it took a couple weeks to be able to make sense of it. Now I don't notice it at all. It's just an exposure / getting-used-to-it thing.

Yep, and within a few months you will find yourself accidentally typing `(if` instead of `if (` in a C-based language.

And then when you go back to a language not based on S-expressions you'll really miss them.

plus an editor with a nice paredit mode.

Having paredit makes me miss LISP when I'm not writing it. It just makes moving around and restructuring code so much easier than it is an a C-based language.

If you learned to unsee the angle brackets in HTML, rest assured, the same would happen in Lisps after a couple weeks.

I use Clojurescript and love it, and also really like Elm. I think the difference is that the Clojurescript community is really on the leading edge of web application development techniques. Elm does it all right, in a similar way, but more basic. In Elm you're also encouraged to develop UI's with no private state and a single Immutable app data structure, but that strategy only gets you so far. Om has the concept of cursors to deal with this, and Om Next will have the evolution of that in a GraphQL type solution which I think is the future.

So basically I'd do any serious development with Clojurescript. Elm is a really cool thing with a small community that's fun to mess around with.

I disagree strongly. I think we'll see more and more developers moving towards Elm's model for exactly the reason you've mentioned. It's simple.

Not only that but having a single source of truth for your app's state, using only pure functions and non-stateful components to render your view allows you to do things like:

  - build a time traveling debugger

  - serialize your app's state and store it in your session so that users see exactly the same view when they log back in

  - trivially do server side rendering

I have yet to be convinced that stateful react/virtual-dom components and/or GraphQL add any tangible benefits in the majority of cases.

Sorry if it wasn't apparent but I agree / advocate strongly for what you mention, and we use that model at my company. If you look at how Om evolved though, it started out with that simple model of passing the state object down to child components. Then cursors were implemented to make it easier to pass around parts of your global state without having components needing to know everything.

GraphQL doesn't mean moving to stateful components, it's just a nicer way to have a singleton data store because instead of chunking and passing a map around, components can declare which data they need. In that system components are still read only, and dispatch to external pure functions.

David Nolen does a great job talking about the trade offs and the evolution of everything in this talk https://www.youtube.com/watch?v=ByNs9TG30E8& I highly recommend it!

> How does clojurescript compare to elm/purescript ?

Clojurescript is a Lisp with dynamic typing, Elm/Purescript are ML dialects where (static) typing is much more important, in Purescript more so than Elm.

I think Elm is interesting because it takes a rather uncompromising FRP approach to web development. Clojurescript does not strive for the same degree of functional fundamentalism.

In the last week, I've done this 2 or 3 times: I've opened some Clojure/CLJS file of mine in a text editor, did a search-and-replace of all open- and close-parens to a single space.

Then I ask if they've seen Python before, to which most say 'yes', and I say, 'there you go, it's Python now'. And I can immediately see the look of surprise when they see that _similarity_, because the similarity is then hard to unsee!

I recommend that trick to any others trying to open minds and challenge preconceived notions in their surroundings. :-)

Side note: I've made the point that nested SQL expressions have many nested parens, too, which we've all done. And that the sum of parens+brackets+braces is similar to other languages, just the distribution is different. Neither of those 2 arguments work as well -- I guess because it's not visual and striking?

Hmm, I feel like such a revelation would break down fairly quickly. What happens when you check equality somewhere, or add two numbers? It's not only parens everywhere (although that is quite the eyesore), other things like prefix notation also makes lispy code unpleasant to read.

This approach reminds me of when people say, hey JS has classes and constructors just like Java does! And then proceed to write Java in JS.

I think it's interesting how much people's aesthetic preferences differ on lisp syntax. Personally, I think the parentheses are visually appealing, but it's probably an acquired taste. Even more than that, I actually now find it much more difficult to parse infix notation for math, equality and so on (and not just operators in syntax-heavy languages like Haskell). Prefix notation, while requiring a different reading style, is incredibly clear and unambiguous as long as no one's gone macro-crazy on you.

PureScript is a very principled pure functional language with a design based on Haskell but adapted to be compatible with JS (it adds extensible records, which are a great fit for JS objects). The type system should really pay itself off in the long run in terms of significantly lowering the cost of refactoring pretty much anything [1]

Additionally, I find that algebraic data types are by far the most natural and complete way to model your data and pattern matching is a powerful way to write succinct and readable code that manipulates that data. The fact that Lisps don't normally come with ADTs (incl. clojure) is a real shame.

PureScript got its own Hoogle [2] recently. It lets you search for functions by stating their argument and result types. This has worked far better for me than any IntelliSense, and its miles ahead compared to rummaging through the docs of a dynamic language hoping to hit a jackpot.

Although the community is small, the speed at which the language and its libraries are moving is quite amazing. For the cases where a package doesn't yet exist, its very easy to write FFI bindings to an existing JS library

On the minus side, there is no "template purescript" yet, and generic deriving is WIP, so some things (like JSON serialization) are still a bit more tedious than absolutely necessary.

The efficiency of the generated code may also be a concern. For example, currying is implemented in such a way that a function call with 3 arguments always generates both the result and 2 temporary JS closures. This can be very costly in certain situations. I find that library documentation is also sometimes lacking, however types and typeclasses do help with that somewhat (the latter only once you learn what each typeclass means).

Finally, something to keep in mind if coming from JavaScript: unless you know Haskell the learning curve for PureScript is far steeper than ClojureScript. Its about as steep as Haskell's. There is a great book available to help with that [3], and almost all of the the knowledge gained there will easily transfer to Haskell. Since PureScript compiles to readable JS, looking at the compiled code made it much easier for me to understand the mechanics of certain features (Eff <-> thunks in particular was quite a revelation) which in turn helped me understand those features

I haven't used ClojureScript much, but it does seem to have more mature, battle-tested persistent data structures (vs purescript-maps) and UI library (om vs purescript-halogen, although halogen seems very promising). There is also a type system (core.typed) which while not as principled as PureScript's, its still amazingly expressive - and since its gradual, you can start fully dynamic and then add types whenever you feel like doing that to parts of the codebase.

[1]: https://www.reddit.com/r/haskell/comments/3e10ea/til_instant... - last paragraph also applies to PureScript

[2]: http://pursuit.purescript.org/

[3]: https://leanpub.com/purescript/

> The fact that Lisps don't normally come with ADTs (incl. clojure) is a real shame.

Was going to say Shen does, but then again I only see pattern matching. Anyway, you might want to check it out:

http://www.shenlanguage.org/

I do agree purescript is awesome though :)

Shen is executable sequent calculus. It's data types are sequents, the conditions specified as arbitrary lisp (shen). Wikipedia seems to classify Shen as having algebraic data types, although it's type system isn't derived from ML or Cateogry Theory.

CLJS is a dynamically typed mostly-functional strict uncurried lisp, Purescript is a very Haskell-inspired languages, Elm is syntactically similar but less welded to Haskell, it's strict, it doesn't implement many features (where clauses, pattern guards) and concepts (e.g. typeclasses, HKT) and strongly focuses on FRP and frontend interactive programming instead.

I've been wanting to get into ClojureScript (to clean my development practices, etc) but I'm wondering if I need to be a Clojure user. Does it make sense for a Javascript developer to just jump straight to ClojureScript?. I've seen PureScript also.

Short answer: yes, jump right in!

Check out David's other blog posts, ClojureScript synonyms[1], and http://clojurescriptmadeeasy.com.

http://himera.herokuapp.com/synonym.html

Clojure and ClojureScript are practically the same language. Both are "hosted", relying on Java and JavaScript for low-level things and providing clean interop for those things.

So, I see no problem with learning either. Especially when you first jump in you are concerned more with the language "proper".

As has been said, they are basically the same language.

In terms of learning, it's probably easier to learn Clojure at the very beginning.

It's simpler to get repl working and get a project building with Clojure right now. Not much simpler, but simpler. And at the very beginning, when the toolchain is magic to you, that matters.

As early as you like you can switch to Clojurescript, learn the handful of small differences and go from there.

It's so simple now:

http://planck.fikesfarm.com/

download, run, and you have a clojurescript repl ready to go on osx at least.

nice thanks

After learning Clojure and ClojureScript I got really excited and was ranting about it to my friend. He skipped Clojure and went straight to ClojureScript and has been having the time of his life.

One thing that's simpler in ClojureScript is that it only has one type of shared mutable reference, the Atom. Atoms are uncoordinated and synchronous. But Clojure has more types, with different behaviour characteristics. So in a way ClojureScript is simpler and could be the introduction.

Does this mean I could use compile and then require clojurescript modules in node? Today I use babel to "transpile" es2015 to es5 and then load those in, could I do something similar with clojurescript?

None of this recent work facilitates compiling ClojureScript into CommonJS modules for use with Node.

But, the opposite is in flight: ClojureScript now has support for depending on libraries that are packaged using the CommonJS system.

> None of this recent work facilitates compiling ClojureScript into CommonJS modules for use with Node.

http://www.matthewstump.com/misc/2012/06/04/writing-nodejs-m...

Should be really very easy to do, you only need to study what cljs compiler emits and the docs for Google Closure Compiler JS libs.

Can someone this to me? It seems that mfikes (parent) comment, and klibertp's comment seem to contradict each other. Not to mention the examples that glenjamin highlighted.

I have heard similar comments in other places, in that it's not possible to package ClojureScript as a NPM module.

Is it because of the size of the module, or something else?

This was already possible, see https://github.com/swannodette/mori or https://github.com/glenjamin/checkers for examples.

I'm curious if ClojureScript uses the original clojure library code (eg, drop, take, map...) or re-implements them? I tried to find something in codebase, seems like it uses original.

A non-trivial portion of Clojure (data structures, compiler) are written in Java. In ClojureScript these are written in Clojure or ClojureScript so there's really not as much to share as it would seem.

This is fantastic. It will be a lot easier for noobs to get their feet wet if they can use use the full language without having to fool around with installing a Java toolchain.

In fact, you no longer even need a full-fledged computer. :)

A while back I enjoyed going through this, also allows you to get a feel without futzing around with tools: http://clojurescriptkoans.com

How far clojurescript is to be clojure/js. By that I mean how close the JVM and JS are semantically. There were discussions about that few months ago, because of differences deep down between platforms builtin types (js integers, etc). Swannodette also explained that cljs and clj don't share as much code as the team would like. How much clojure code would run correctly as-is on cljs ? Without having to use specific reader conditionals.

Thanks, for all the fish ( by fish I mean awesome code ) @swannodette and team! Can't wait to use Om Next.

Also looking forward to seeing some code examples of Demand-Driven Architecture[0]. Does anyone know of any edifying clojure code examples?

[0] http://www.infoq.com/presentations/domain-driven-architectur...

I wonder if one of my favorite libraries for Clojure, Anglican [1] a probabilistic programming language, can be ported to ClojureScript without much effort. One could do amazing things with this.

Keep up the good work. Clojure & ClojureScript are awesome.

[1] http://www.robots.ox.ac.uk/~fwood/anglican/

If your library is written in pure Clojure (no reliance on Java) it's fairly trivial with Clojure 1.7's Conditional Readers.

Here's an example of a repository that I've repurposed to compile both Clojure + Clojurescript.

https://github.com/rymndhng/clj-diff/commits/add-cljc-redo

How does the output of the ClojureScript-compiler-in-ClojureScript compare to the real thing? Do you lose the Google Closure optimizations, and if so, does that mean it isn't a good idea to compile your web app using the ClojureScript compiler running in Node (for example)? It's nice if you can compile ClojureScript on a machine without Java.

Correct; you do not get the benefit of Google Closure when using the bootstrapped compiler. The JVM-based compiler will continue to be the best way to compile ClojureScript projects for production.

This is a huge milestone! I had a problem I was solving with Clojurescript six months ago and having access to the reader would have solved it perfectly, with lisp elegance. Instead I had to make a mess.

I think this opens huge computer science educational opportunities. Imagine a LOGO DSL inside your browser. With all the power of cljs. I am very excited!

Wow, this is quite amazing! Does this also mean that we will be able to write macros in Clojurescript?

Are there any plans to use mostly the same backend for Clojure and Clojurescript? It seems that since the languages are almost the same, it's only the code generation part that would be different?

Yes and no. You can write macros for which the source is ClojureScript, but they need to be kept separate. Here is a stab I took at explaining that subject: http://blog.fikesfarm.com/posts/2015-06-19-portable-macro-mu...

There are no plans for the same backend AFAIK.

Using the same code for the backend is not realistic since Clojure leverages JVM libraries. A new ClojureScript backend ecosystem would need to start wrapping Node libraris to reach Clojure parity

I did this late last year for a traffic analytics backend. As you said, I had to wrap a lot of Node.js libraries, and then wire them up with core.async.

It was a fun project, and I learned a lot, but I ended up reimplementing the whole thing in Clojure+Java because performance and DB access was so much better in this use case. Having only a single event thread kills Node relative to the top JVM servers.

I think an article outlining the details behind the non-trivial example, specifically on configuring how library names are resolved and how it can be modified dynamically, would be instructive.

The source code for the post is available[1], as is the macro[2] which is loaded via XHR.

The short of it is that a load function is passed to `cljs/compile-str`, which provides the mapping. In this case, it doesn't bother to do any mapping because it knows that it is loading `bar.core`. However, `bar-url` could instead be in a map from the namespace to the URL. Surprisingly simple, actually!

1. https://github.com/swannodette/swannodette.github.com/blob/m... 2. http://swannodette.github.io/assets/cljs/bar/core.cljs

Awesome!!! Now we want Clojure in Clojure :)

Doesn’t this imply the development of a JVM written in Clojure?

Nah, A Clojure program which translated Clojure code to JVM bytecode would be a good start.

Like this? https://github.com/clojure/tools.emitter.jvm

When will the module support be released? Soon I hope! Thanks for the great work on all this.

Module support is already in 1.7.28 however it requires a corresponding release of the Google Closure Compiler.

For some reason I feel like ClojureScript starts to transform into a new Clojure dialect (which runs on node.js and not on a JVM). Am I correct?

ClojureScript will always run on the JVM. It's just that from now on it will also always run on JavaScript.

Here is some good exposition on that matter: https://github.com/clojure/clojurescript/wiki/Bootstrapped-C...

Hm. This allows putting functions into edn that gets sent to the browser now, then doesn't it? Since ClojureScript can eval now?

If you put the function source into edn and the function doesn't depend on anything but core constructs, it would be nearly trivial to eval using the new cljs.js.

If those functions need to call into other ClojureScript namespaces that you've already compiled into your code then the analysis cache would need to be populated so that eval can operate properly.

For reference, here is cljs.js/eval: https://github.com/clojure/clojurescript/blob/v1.7/src/main/...

Finally, thanks god

Reply on news.ycombinator.com