Hacker News Reader

Comment by cromwellian | Hacker News Reader

"The trend started by minifiers like UglifyJS and continued by transpilers like Babel will only accelerate."

Those were predated by Closure Compiler, which was started with Gmail, and Closure Compiler in some ways still doesn't have an equal. Besides being a fully optimizing compiler, it also has a module system for code splitting, and optimization passes designed to move code at the method/property level from initially loaded code into deferred loaded code if it is detected to only be used later.

You can see this in action with photos.google.com which splits up every UI component, model, and controller into a dependency tree of modules, which are aggressively optimized. When an action needs to fire, only the transitive dependency of reachable code needed to handle it is loaded.

The closest external version of this is Malte Ubl's"Splittables" for Closure Compiler (https://medium.com/@cramforce/introducing-splittable-1c882ba...)

View on HN

This seems to be something of a trend in the software library and tool world:

A problem emerges. Some early, sharp thinkers create tools which attack the problem in a high level, abstract, powerful way... but also with the rough edges that come with being early. These tools get a few adopters, but not that many because it is difficult to see the value versus the obvious rough edges.

Then other sharp thinkers, put off by those rough edges, create N new waves of tools which attack the problem less effectively, but more accessibly.

Then eventually, some years later, common practice of the waves of tools will circle back around and start doing the things that that early rave started.

It's easy to criticize the "daily new JavaScript framework" phenomenon along these lines, but I don't think it is specific to JavaScript, nor specific to 2017, and not even specific to software development. It's just part of how we humans work.

Reading your comment reminded me of a classic piece [1] about a similar phenomenon.

[1] http://discuss.joelonsoftware.com/default.asp?joel.3.219431....

Somewhat relatedly - when a new technology wave gets started, it's usually proprietary, because there are big competitive advantages that can be had by being the only one with possession of a new technology. As details leak out and enterprising independent hackers get interested, commodity or open-source clones emerge, and they get much wider distribution simply because they're cheap or free. The original innovator is caught flat-footed, loses mindshare, and eventually has to adopt the open-source solution when it becomes dominant in the open-source world.

Altair -> Apple -> IBM -> Dell & clones. UNIX -> System V -> BSD -> Linux. MapReduce -> Hadoop. GMail -> Closure Compiler -> Traceur -> Babel. Google datacenters -> AWS -> Docker -> Kubernetes.

This is a very strange example. Gmail, Closure, and Traceur are all Google products.

And Closure and Traceur are both open source.

And Babel's the one people actually use.

Within the progression, each new generation is decreasingly tied to a proprietary, revenue-generating product, and increasingly intended for outside consumption. So GMail was completely proprietary and closed off from the outside world. The JS library, compiler, and optimizer from GMail was spun off into Closure, which was widely used across Google products and finally open-sourced 5ish years later, but never got widespread external use because it's very tightly tied to the style of JS development within Google. The idea of JS transpilation was proven by Closure, and Traceur was developed from the outset to be open-sourced and not tied to Google products - but it was developed by a member of the Chrome team, and so the developer perspective was that of how internal Google engineers think, and the development process wasn't really inclusive of outside engineers. Finally, Babel was developed entirely outside Google, based on ideas that had been proven out and were widely disseminated, but it was developed from the start with an open community process.

I think some of this might be that selection bias. Closure compiler is popular, plenty of not-so-sexy projects have used it since its inception to a great degree of success. I seem to see it with some consistency in the enterprise world. It's never been popular with the HN / startup crowd.

As someone who had some amateur experience with C++ roughly 10 years ago I felt it was sort of deja vu with JavaScript. Transpilers, aggressive optimization, automation scripts it's nothing that was too unfamiliar with me.

Webpack is roughly a library linker. Babel is like GCC Make. Gruntfile ~= makefile. I think it's coming of age but the JS ecosystem is still undergoing growing pains, plus it can be overwhelming to newbies. People are paralyzed by too much choice. Having IDEs like Webstorm would be good at least taming or organizing workflow better.

Babel is more like cfront, transpiling from a superset to an older base set of language features. Edit: Maybe you meant to compare it to Macro Pre-Processing, which is often set in makefiles. But it's a shim loader.

This was also predated by the Opa "framework" which is indeed a compiler for the web platform, announced in 2009 and released a bit later.

At the time, people didn't want to hear about a new language so we had to hide behind a framework. But one of the ideas was that there is no sense in bundling libraries at runtime instead of generating exactly the application code that should run.

Cf. http://opalang.org (disclaimer: I was the project creator)

Yup, Opa is pretty nice. It just highlights the differences between "compilers" and "transpilers" which came up in another thread recently. There's a number of transpilers that are fairly simplistic transforms of source languages, or JS->JS translation (lowering ES7/6 to ES5, etc) which do limited whole world optimization, and mostly consist of peephole passes.

Closure, Opa, GWT, Dart, and a few others (ScalaJS?) run fully optimizing compiler passes like a traditional old school compiler. For example, IIRC, Dart transforms (or used to) to Hydrogen IR/SSA internally, runs lots of optimizations, and then transforms back before JS output.

This is not to bash transpilers, but I don't consider Webpack/Babel/Uglify in the same category.

I was just thinking about Opa the other day - it looked nice but I never used it. People were worried about debugging through the layers of abstraction. Is it still "alive"? You write in the past tense...

It is not really alive as of now. But there is still space for a full stack language and Opa is relevant today. MLstate, the startup behind Opa, shifted to secure communication platforms (built with Opa) and was acquired last year but without Opa itself.

Many things we did became hype later (implemented in OCaml, a functional language, JSX before React - I know Jordan played with Opa before building React) and we still have bits that are missing is today's stacks. The ideal next step would be to join a foundation so that development on the project could resume.

Tierless programming languages did not started nor ended with Opa. Eliom is very well alive and kicking (disclaimer: I'm finishing my PHD on the topic), as well as Hop, Split.js, Ur/Web, Links, Websharper, ...

I was always very disappointed by the fact that the inner workings of Opa were never described in any way. I can't even really cite it in my PHD thesis, because the documentation for the early (and more interesting, imho) versions has completely disappeared.

The first I saw was Curl from MIT:

https://en.m.wikipedia.org/wiki/Curl_(programming_language)

What is present in Opa that is missing in other stacks?

The compiler for GWT has done a lot of this for years (and it's functionality might have fed into closure). It's arguably different because it compiles from Java to Javascript as a first pass. It then does a lot of full-program optimization, including dead code elimination, lifting virtual method calls to static, inlining, reordering code to compress better, and analyzing the call graph to split code. (You have to be explicit about the split points in the Java code, so the analysis is mainly for doing the code splitting and for coalescing components that are are small or would force a lot of code into the common segment.)

I believe GWT is moving on to closure for the latter optimization passes in future releases.

Closure Compiler is very, very good at producing small code.

It also takes a fair bit of manual effort unless it's handled for you (e.g. ClojureScript, Scala.js).

Output modules in particular demand some very particular attention.

Douglas Crockford is the guy who pushed JavaScript into this world. He wrote the first jslint which put JavaScript in the build pipeline on it's own. Also the YUI compressor and minifier are old enough to be deprecated.

The Closure Compiler is the only reason I have Java installed, it's that good. And IMHO, a lot of the bloated JS projects nowadays would really profit from its dead code removal...

In case you haven't heard, you can now use it without Java https://developers.googleblog.com/2016/08/closure-compiler-i...

That's actually really amazing - and I didn't know that, thank you!

It's extremely slow compared to the Java version though.

I think it's about 2x slower.

they're the same in name only. but not in perf and not in output.

https://github.com/Rich-Harris/butternut/issues/35

I don't see where in that issue it says output changes from java to js? Other than the runtime option

The Google Closure compiler and library are used as targets by the ClojureScript compiler https://clojurescript.org/about/closure

I remember the first publicly available being Dojo's minifier which used Mozilla's Rhino to determine which symbols could be safely obfuscated. It's primitive by today's standards, but it was a huge step forward at the time.

Those features are supported in Webpack and other bundlers.

Code splitting: https://webpack.js.org/guides/code-splitting/

Tree shaking: https://webpack.js.org/guides/tree-shaking/

It's not quite the same. It has some aspects of Google's module system in terms of de-duping (our internal version automatically splits up all code referenced in more than one place into a tree of synthetic modules) But Closure's optimizations go far beyond the "tree shaking" described here, which is really about pruning unused module dependencies.

Closure does pruning of methods, properties, etc at a fine grained level, moves code at a per-method level between modules, it computes whether functions have side effects and elides unused side-effect-free code, it does type-based optimizations (https://github.com/google/closure-compiler/wiki/Type-Based-P...), the list goes on and on. It's really like having GCC or LLVM for Javascript code. That's one reason why it's slow, because it does far more than the transpilers and minimizers out there.

(As many in another, GWT does many of the same things, being a fully optimizing true compiler)

I want to add that you can easily write your own compile pass for the closure compiler as well. All the optimisations described above are just compile passes.

Only in a very primitive way.

https://github.com/webpack/webpack/issues/2867 - "Tree shaking completely broken?"

It turns out that it basically supported "skip loading of unnecessary ES6 modules", so kind of like a "warn: unused import", but turned into a silence and no-op.

Now with Uglify getting the "pure" annotation, and libraries sprinkling that onto their code, it'll help a bit.

React moved some parts from Uglify to Closure recently. They're only compiling using "simple" ("none" and "advanced" are the other options), but they're still seeing benefits: https://github.com/facebook/react/pull/10236

In the beginning there was GWT. But along with Java, GWT has a steep learning curve so its appeal is limited.

Reply on news.ycombinator.com