dherman

All abstractions can make your code unreadable; the key, as always, is to create good abstractions and document them clearly. Macros are syntactic abstractions. It's just as important to document them as it is for functions or objects. Of course, if you just have a little local macro that you're using for convenience, looking at the implementation may be sufficient. But when you write a macro that you want to share from a module, rather than requiring your clients to read the implementation, you document the syntax and the semantics, just like you would if you were writing a separate language.

But by having macros directly in JS, instead of having to use a whole language that compiles to JS, you can combine syntactic features from different sources. For example, right now there's no way to use one feature you like from CoffeeScript with another feature you like from TypeScript. You just can't combine them. But with macros, you could import two different syntaxes from two different libraries and use them in the same code.

On top of that, if we actually had macros in a future version of the standard, you wouldn't even have to precompile the macros offline, and you wouldn't need a preprocessing step at all. (For latency purposes, you might want to preprocess macros offline as an optimization. But for development, not having to do a preprocessing step is more convenient.)

Dave

https://developers.google.com/native-client/

> or like with Brendan's slide that desaturates a picture of salt as if (P)NaCl is going to come for your children in the night.

Really now. I created that slide, taking the picture from Google's own NaCl web site:

I zoomed it in because I thought it looked pretty. Let's all just take a deep breath.

Dave

> Is it just me or does the proposed ES6-standard smell of heavy feature creep?

I've been on TC39 for years and the vast majority of what we do is cut.

> we get classes _and_ modules _and_ typed objects.

Classes are syntactic sugar for one of the most common dynamic patterns in JavaScript: object factories. Modules are static collections of code. Very different things, especially in a dynamic language like JS. Typed objects are not even remotely related; they're a low-level API for efficiently working with structured binary data.

> same with macros and codegens.

As Yehuda says, macros are not part of ES6 but rather an experiment at http://sweetjs.org to consider for future editions. I don't know what you mean by "codegens." Brendan's slides weren't talking about a feature, but rather an existing usage pattern that we need to serve in additional to human code-writers.

ES6 actually does a remarkable job covering many use cases with small modifications that smoothly integrate with the existing language. You might want to try experimenting with some of these features, many of which are implemented partly or completely in SpiderMonkey (as Brendan's latest blog post details). Most of the features are small improvements to paper cuts, and as everyone I've ever spoken to who's written SpiderMonkey-specific JavaScript attests (for example, in the Firefox front-end or in addons), they make your life so much nicer. I'm particularly thinking of let and destructuring, and I suspect parameter defaults and rest-arguments will be hugely popular as well. The few bigger features, especially modules and generators, are for addressing the most important gaps in JS.

Dave

Though the expression version of => comes pretty close to TCP.

Dave

> The presentation focused on what it perceived as missing features: structs (seriously?), classes, modules, syntactic sugar, macros, etc.

The nature of a widely-used technology is that you can't remove features, you can only add. And yet adding features causes an increase in complexity. So what to do? The answer is to add features judiciously: prefer general features that cover a wide array of use cases and can provide better ways to do things that the existing features don't do or don't do well. (But also avoid over-general features that destroy important invariants -- for example, just say no to call/cc or threads.)

> But the huge gaping holes in Javascript are not missing features. They are fundamental errors in the language. Things like ==, numbers as strings, eval, incorrect definitions of false, semicolon insertion, and -- heaven help us all -- improper lexical scoping.

ES6 -- and potentially down the road, macros -- are paving paths to fix many of these problems you mention, and other important problems besides (e.g., callback hell). Lexical scoping is partially improved with ES5's "use strict" and further improved with ES6 modules. Block scoping finally exists thanks to `let`. ES6's module loaders allow translation hooks to enable dialects or alternative languages to be run in-browser (which you can do with preprocessing and build steps and directly, but using them in-language streamlines the "shift-reloadability" development experience). Module loaders also provide a saner eval, which allows you to completely sandbox the evaluated code. Macros could even allow rebinding operators like `==` to have cleaned-up semantics. We intend to try this out with sweet.js, building something like "restrict mode" (http://restrictmode.org) as a module you can import.

Dave

I'd forgotten about the whole Lisp tradition of expanding + evaluating one form at a time -- once again betraying my Scheme biases. :) That's making it hard for me to think about how to compare the two. I'm really not familiar enough with the one-form-at-a-time approach.

Anyway, my takeaways here are:

- I shouldn't've said anything about Clojure specifically, because a) the design space is different and b) they have some form of hygiene-like something-or-other that I don't know enough about. Gotta go study them!

- I still don't know how to do hygiene in Scheme-like languages any other way than the approaches I've seen, but I do agree they're more complicated than I wish they were.

Dave

Interesting. I don't see how this works. I wonder, is it different if you do this?

    (let [thing "outer thing"]
      (defmacro foo [prefix]
          `(str ~prefix thing))
      (let [thing "inner thing"]
          (prn (foo "should say outer thing: "))))

Dave

> Ok, so are you are concerned with the case where the person defining the macro uses a symbol from the namespace of the person using the macro but that symbol has been rebound by the user inside of a let surrounding the aforementioned usage of the macro?

> If so, that requires a cyclic module dependency...

Not necessarily. For example (forgive the Scheme syntax), all in one module:

    (define thing "outer thing")
    ;; define-inline is the above macro-defining-macro
    (define-inline (foo prefix)
      (string-append prefix thing))
    (let ((thing "inner thing"))
      (foo "should say outer thing: "))

> Clojure does not have these issues: when the macro is called, the symbols are already attributed with the full namespace qualification, and usage of quasi-quote inside of the macro definition will also apply namespace qualification to variables local to the definition of the macro; you have to go out of your way to break this.

Again, not a Clojure expert, but a namespace is coarser-grained than individual local scopes, right? The problem I'm talking about is when you have a local variable inside a nested scope (e.g. inside a `let`). If this is not named by a namespace, then you would still get collisions.

Regardless, Clojure's approach seems to be much closer in spirit to a hygienic macro system: it attempts to get scoping correct by default, and allows you to intentionally capture.

> You should spend more time looking into it before claiming to people that it doesn't work correctly; you could easily have just said "that's a good question, we'll look into that after the talk" rather than telling the person that Clojure wasn't as good.

Fair enough as far as it goes. I did react snappily, but you didn't hear the offline conversation (this whole thing was a dialog at my office with a friend and colleague, incidentally) where I said "I'm not entirely sure, but I believe there are things you just can't express with systems like Clojure's." And we concluded, just as you reprimanded me to do, that we should look into it further when we have time.

Dave

dhermanon Hey Yahoo, You’re Optimizing the Wrong Thing

That was me. :) Caveat: I know more about hygienic macros than I do about Clojure, so I'm not in a position to critique Clojure specifically.

Hygiene is (roughly speaking) about getting scope right by default but has never been about forcing it on the programmer. Moreover, there are two components to it, only one of which is easy to achieve in an unhygienic system. It's easy to ensure your macro renames introduced /bindings/ by using gensym. But if your macro introduces /references/ to existing variables, it's very hard to protect against those references getting captured at the site where clients call your macro.

I believe in Clojure they get around this for some cases by letting you fully qualify a reference to a library binding, for example. But what if your macro wants to refer to a variable that's local to it? Such as an unexported library function, or simply a local variable. Again, I don't know if Clojure has an answer to this.

One concrete example: write a `define-inline` macro-defining-macro. At the call site, a user might write

    (define some-local-variable /* something */)
    (define-inline (foo x)
      (+ x some-local-variable))

In an unhygienic system, they should first of all fully-qualify the `+` to be safe (yuck!) whereas a hygienic system just gets that right by default. But more critically, how can they be sure that some client of `foo` won't write:

    (define some-local-variable "something else")
    (foo 42)

Not only will that break, it's not clear how to fix it. A hygienic system gets this right.

I do agree that the state of the art in hygienic macros is too complex. I just haven't seen another system that makes this kind of thing work. But I would like to experiment with Clojure's macros more to see if they have an answer.

Dave

[Disclaimer: I have decades of first-hand knowledge of Hilary's awesomeness, going back to when we were CS students together in college. So yeah, I'm defending my friend.]

I'd like to ask you to think twice before publicly questioning someone's credentials like this. Whatever your intentions, picking on someone's CV just because of a blog post you disagree with is not only rude, but it sends a message -- particularly to women in tech -- that if they speak publicly, if they offer up their opinion, they will be attacked not about the content of their point but about their competence to speak at all. I believe this kind of attack has real consequences on our field, and I would urge everyone to show everyone the respect they'd want for themselves.

Dave

dhermanon Write any javascript code with just these characters: ()[]{}+

Unfortunately, his tricks no longer work in current JS engines; it relies on using

  [].sort.call()

which I believe used to return the global object but now throws an exception.

AFAICT, you need to add {} to make this work in current JS engines.

Dave

dhermanon Write any javascript code with just these characters: ()[]{}+

I think if you wanted to make this really robust, you could use more techniques to beef it up.

One technique would be to store verbose or commonly-used string constants in accessible locations like Array.prototype.f. Then you could access, say, the string "prototype" by simply writing

    [][(![]+[])[+[]]]

Once you build up a little scratch storage of the most common or hard-to-encode strings, everything starts getting orders of magnitude smaller.

(Technically, this means that you're polluting the space shared with the program being encoded, so for everything to work the program can't make use of it. But that's a pretty simple invariant to ask of the input program: "don't get or set the 'f' property of arrays.")

Another technique would be to break up large statements into smaller substatements, to avoid fixed limits of JS engines on statement size. You can always avoid semicolons, since ASI is guaranteed to work if you start your statement with a !.

Dave

dhermanon Write any javascript code with just these characters: ()[]{}+

PS I flipped the logic in my explanation; whitespace coerces to 0 and letters coerce to NaN. Which is why filter removes the whitespace and not the letters.

dhermanon Write any javascript code with just these characters: ()[]{}+

Funky! Luckily that bug doesn't interfere with this trick, since it's only relying on pretty-printing integers.

Dave

dhermanon Write any javascript code with just these characters: ()[]{}+

Arg, scooped! I was working on this exact same thing! :D

Since you've beat me to it, let me offer up a couple additional tricks you might want to use. If you want to make this completely independent of browser API's, you can eliminate the dependence on window.location (or atob/btoa as the sla.ckers.org poster did).

Trick #1 is to get the letter "S".

You can extract this from the source code of the String constructor, but you want to be careful to make this as portable as possible. The ES spec doesn't mandate much about the results of Function.prototype.toString, although it "suggests" that it should be in the form of a FunctionDeclaration. In practice you can count on it starting with [whitespace] "function" [whitespace] [function name]. So how to eliminate the whitespace?

For this, we can make use of JS's broken isNaN global function, which coerces its argument to a number before doing its test. It just so happens that whitespace coerces to NaN, whereas alphabetical characters coerce to 0. So isNaN is just the predicate we need to strip out the whitespace characters. So we can reliably get the string "S" from:

[].slice.call(String+"").filter(isNaN)[8]

Of course, to get isNaN you need the Function("return isNaN")() trick, and you know how the rest of the encoding works.

Trick #2 then lets you get any lowercase letter, in particular "p".

For this, we can make use of the fact that toString on a number allows you to pick a radix other than 2, 8, 10, or 16. Again, the ES spec doesn't mandate this, but in practice it's widely implemented, and the spec does say that if you implement it its behavior needs to be the proper generalization of the other radices. So we can get things like:

(25).toString(26) // "p"

(17).toString(18) // "h"

(22).toString(23) // "m"

and other hard-to-achieve letters.

But once you've got "p", you're home free with escape and unescape, as you said in your post.

Dave

dhermanon Mechanized LambdaJS

That's at least partly because SpiderMonkey supports a superset of the ES5 language, including Mozilla-specific extensions and candidate features for future versions of the standard.

Dave

dhermanon Mechanized LambdaJS

It's not so much a reduced subset as a kernel language: the full ECMAScript Edition 5 language can be translated down to the kernel language, so in principle they can actually model the behavior of any program that the language defined by the standard admits.

There have been a few attempts to formalize the semantics of JS, but the lambdaJS approach is the most practical: by factoring out the incidental details (the "syntactic sugar") as a translation ("desugaring") to a kernel language, any work you do on the formal semantics of the language only has to focus on the smaller kernel.

This is also a really practical way to approach building tools for JS, like static analysis: if you design a single desugaring pass as a translation, then you can build a whole bunch of tools on the kernel language. Same idea roughly as the standard compilers approach of translating to an IR.

dhermanon Twitter crashes itself with commas

This headline is misleading, especially given how much people love to argue about semicolon insertion. This has nothing to do with semicolon insertion. It's just about an artificial constant limit in how large a certain type of parse node (comma expression) can get in Opera's JavaScript parser. (Of course, Twitter should have tested in Opera!)

    It's easy to say "X is not cheap, Y is slow".

Yeah, I probably wouldn't have worded it that way. I think the better way to look at it is, it's hard to know how to predict the performance of JavaScript, given the complex, dynamic, heuristic optimizations performed by modern engines. This project is better thought of as an attempt to build a dialect of JS that can be more easily tuned for performance, because its performance model is simpler than that of JS.

If I were going to try to make that claim more precise, I might do some experiments to demonstrate the high variance of object performance in modern JS engines.

    I could see the value in certain niche situations...

That's really the idea. This isn't meant to be an alternative to JS, but rather a tool for performance-sensitive kernels. Interoperability will therefore be key, because it has to be easy to write just a small component of a larger app in this dialect while smoothly integrating into the rest of the app.

    ...but I would really like to see the performance characteristics explored more fully.

Perhaps. I'm skeptical of our ability to accurately measure general claims like "objects are slow" or "X is easier to program in than Y" or "*JS is easier to performance tune than JS." But I do think it's easier to evaluate those claims, at least informally, after you've built something you can experiment with. Hence this experiment!

http://blog.mrale.ph/post/12396216081/the-trap-of-the-perfor...

JS JITs are wonders of modern engineering, for sure. But there's a weird paradox in that as they get smarter, it doesn't get easier to write optimized code, it actually gets harder! Nobody really understands how to write optimizable code. Nor, in some sense, should they. But I think there's a real need for subset languages with more predictable performance models than JavaScript, that have compilers tuned for the kinds of optimization done by modern JS engines.

One nice description of this general problem was by Slava Egorov:

Just a temporary name. We're tossing around name ideas. Michael's a true hacker. He writes code first, asks questions later. I'm more useless, so I like to think about names. ;-)

Sure! Right now there's just Michael's GitHub repo (this is literally only a few weeks old) but I've created an IRC channel -- join us at #lljs on irc.mozilla.org. You may have better luck getting attention during the work week, though. ;-)