Clamato: A Smalltalk Dialect for Javascript | Hacker News Reader

Clamato: A Smalltalk Dialect for Javascript

clamato.net

58 pointsBigZaphod38 comments

The use of JS as an object code seems to be a big trend lately. This makes me wonder:

(1) How suitable is JS for use as an object code?

(2) Is this trend taken into account by those designing the future of JS?

I can't answer either of these very well myself, (although for #1 I speculate that it's prototypical nature makes it easier to target than a class-based language might be), but I think it's primarily targeted by language experimenters because it's incredibly ubiquitous and easy to demo to others (there's nothing new to install) and the coming generation of JS engines are finally using a lot of cutting edge techniques to drastically improve performance which should make their work more practical in the near future.

How suitable is JS for use as an object code?

Highly. I think it's a much nicer language to compile to than to write by hand (although JS is a pretty good language), especially if you want to use a very high-level language and have your code run in a web browser. We compile to JS from Common Lisp using a set of macros built on top of the excellent Parenscript library. We've built things up incrementally to the point that we're now able to write most of our app in a subset of Common Lisp that compiles to efficient JS. The other day I took some Lisp code I wrote a year ago and generated JS from it without having to change it. That's the first time that's happened. (We'll probably open-source this eventually as a layer on top of Parenscript.)

A big advantage of this approach over JS libraries is that you can emit quite low-level JS, so you don't have to pay for all that abstraction at runtime.

Having said the above, I'll partly take it back. JS isn't completely object code in the sense that you can generate it and forget about it. You need to be able to debug it in the web browser, and that means you need to be able to read it. In that sense, it's not really object code.

"Having said the above, I'll partly take it back. JS isn't completely object code in the sense that you can generate it and forget about it. You need to be able to debug it in the web browser, and that means you need to be able to read it. In that sense, it's not really object code."

I'm pretty sure it must be possible to 1. preserve the "semantic trace" of compilation from high-level Lisp to low-level JS and then 2. write a high-level debugger that, given a piece of object-code that went wrong, points to the high-level code that generated it.

A complete solution might be intractable but even a partial solution could greatly help debugging.

I compile from ECMAScript 4 to JS, and what I do is create a separate mapping file, where line/char pos for every token in the generated file is mapped to file/line/char in the source.

When the JS engine throws a runtime error it typically (depending on the engine) includes line/char pos for the point of error. A wrapper then extract line/char from the error message, finds the corresponding position in the original source, and displays that also. This greatly helps debugging runtime errors.

It is not perfect, since some engines (e.g. Rhino) doesn't include position for runtime errors.

(I was offline for a few days, but want to reply anyway.) My partner and I were just discussing this the other day: write a high-level debugger that, given a piece of object-code that went wrong, points to the high-level code that generated it. There are a lot of interesting tools one could build in this space. For example, a Slime-style REPL that provides Firebug-like access to JS in the browser, but via Parenscript forms, would be a really nice addition. As for your debugger suggestion, I would love to have such a thing. But first, I'd settle for having such a thing in the Lisp environment alone - it's my biggest pet peeve about CL that the debugger situation is so bad.

I can't speak to the second point, but to respond to the first: it's very well suited.

I've built two primary compiler projects targeting JS, a Python->JS compiler and a 6502 emulator that dynarec's to JS. Both were easy to build and its flexible nature made everything work out very well. I dislike writing JS myself (just don't dig the syntax), but writing compilers backending to it works out quite well.

I have to disagree with the other commenters as to its suitability as general object code.

I find it too high-level, with insufficient control over the lower-level details. For Skulpt (my Python-in-JS implementation) it works "ok" because JS and Python are similar in some respects, so some things map pretty well. But, for other things, like implementing longs (aka bigint), or implementing exceptions and attribute lookup faithfully to Python's semantics, it requires overhead that I consider unnecessary. Control structures from more flexible languages would require extensive hoop-jumping too.

Really, JVM-in-browser was the "right" solution, but the implementation of that is obviously horrible and impractical.

Not sure about #2, I guess it's a chicken-and-egg problem. With Google compiling from Java there should be some motivation to improve that use-case.

(1) Depends on how different the source language is from the semantics of JS. JS is a high-level language and doesn't have e.g. goto which would make it easy to implement different execution models. If you don't want to create your own interpreter in JS (which would be an alternative to code generation), you have to be manage with functions (and exceptions). If the source language require more powerful constructs like goto or continuations, you are out of luck. (Theoretically you could gain some power by compiling to a continuation-passing style, but since JS engines doesnt have TCO, I dont think this would work in reality.

The scoping rules combined with the distinction between expressions and statements means that it is harder to generate composable snippets. For example you cannot just generate a small "context free" code snippet with a local variable - the scope of a variable is always the function, so you need either to mangle the name to create a unique variable, or create an inline function and call it - but that changes the semantics of return, which again is a hindrance to composability.

I would definitely prefer something like scheme as a compilation target, but it could be a lot worse. Imagine compiling to VBScript :-)

For my particular project which compiles ES5/Harmony to JavaScript it works quite well, but those languages are based on JavaScript anyway, so that is almost cheating.

(2) AFAIK the working group are a aware of the interest in JS a compilation target, but are not at the time considering adding features specifically to make it more suitable for that purpose.

Code: http://bitbucket.org/avibryant/clamato

I hate you! I was looking for a Smalltalk->Javascript thing before I started my current project, but I couldn't find one other than st2js. Arrrgghhhhhhafuhoasdf

asdfiojafdiojafdsooijfa

Anyway, thanks, this is awesome.

It's not really in the shape I would have wanted for a public release... kudos to the submitter for finding the website, I guess :). But if you find it useful, great.

I always thought Smalltalk seemed like a closer match to JS than Scheme, if you got rid of first-class message passing. How much overhead is there? For example in Objective-J in reasonably tight loops, I usually try to avoid message passing or using the Cappuccino types if I can. JS is already pretty slow in IE and older browsers, and UI updates can become noticeably slower with more overhead.

There's no overhead, in the sense that one Smalltalk message send maps to one Javascript method call. However, Smalltalk tends to use more, smaller methods, and more blocks, so there's some stylistic overhead that will make idiomatic Clamato slower than idiomatic Javascript.

Does it support doesNotUnderstand?

Nope. The goal here is not to duplicate every aspect of Smalltalk, only the ones that map easily to Javascript.

I thought Clamato was primarily a Mexican thing for Chelada (you can get pre-mixed Clamato and Budweiser in Mexico and Mexican areas in Chicago and other cities).

I thought you Canadians were better than that. shakes head

To the best of my recollection Clamato juice was created for the first time in Boston or New Jersey, but popularized via the Caesar in Calgary (like a Bloody Mary with Clamato instead of tomato).

Either way, it's a prevalent drink in Canada and difficult or impossible to find elsewhere.

No need for recollections when we have Wikipedia! http://en.wikipedia.org/wiki/Clamato Memory is so fallible anyway - why even use it? :P

Sorry Avi - it's just so darned compelling... :P

You can do penance by submitting some patches.

There are several departures from Smalltalk’s syntax and semantics. The most notable of these are: no explicit returns, 0-based indexing, no metaclass hierarchy, special syntax for instance variables.

I can live with no metaclasses. Special syntax for instance variables isn't that much of a problem. 0-based indexing -- this should be easy to take care of with your own collection classes.

However, no explicit returns -- this one kills me.

I guess this has to do with Javascript.

EDIT: JS has exception handling, so this could be used to implement explicit returns.

The problem with using exception handling for explicit returns is that you'd need to put a try/catch block at the top level of every function. That seems yuck. Or am I missing something?

Exactly. For those times that you really need them, I added a #return: method that you can use explicitly:

self return: [:ret | .... ret value: foo ...].

But you only pay the cost of the try/catch/throw when you actually use it.

FWIW, I've found that the lack of explicit returns really isn't a problem, at least in the code I've written and ported so far.

I've been offline for a few days and just read this:

But you only pay the cost of the try/catch/throw when you actually use it.

Do you mean that the existence of an explicit return is recognized at compile time, so only the functions that actually contain one need to be wrapped in a try-catch? Now that I think of it, that seems like the best way to do it. I've wanted to be able to compile return-from into JS for a while...

I was thinking of the compiler doing it for me. I'm not interested in writing new code. I'm interested in porting a lot of old code. It would be best if I could get it to "just work" modulo an automated translation.

I suggest some relization of explicit conditional return for Clamato: http://sites.google.com/site/abbatfilestore/clamato/cookbook...

Also, please join to Clamato google group: http://groups.google.com/group/clamato-smalltalk/

After all these years reading about it, the clamato tutorial was the first time I got to actually try out a code browser interface. Very cool.

"a browser fit for the gods" <G>

The current Ruby to JS project: http://www.ntecs.de/projects.html

After some recent ambitious JS development, it's become clear to me that the language is just unsuited to large scale development and no library is going to change that.

As such, I am really keen on these compiler projects, but I think it's a mistake to avoid language features because they are difficult to implement in JS. Syntax is nice, but it's those language features that are really needed.

It depends on your goals. For me, it's not so much about difficult to implement as whether the generated JS is idiomatic enough for the various optimizing JS implementations (V8, SquirrelFish, TraceMonkey) to handle it. With Clamato, we've shied away from implementing language features until we could see how to do them in a way that wouldn't hurt optimization (the only exception I can think of is the Smalltalk cascade syntax, which is implemented naively and slowly for now, but it's obvious that with a bit more work it could be sped up).

Wow, slick! Browser for environment/introspection is definitely a killer feature. What does "save/commit" do? Is there a git-y thing behind it? It'd be nice to be able to have it auto-push to github et al.

I get a generic "Error committing to counter.st" when I try to follow the tutorial though right now, using Chromium nightly on Ubuntu.

(Trying to think of a project that could be called Caesar... ;)

edit: oops, can't read: the browser's already called Caesar.

Save modifies the method in the current environment. Commit does an HTTP PUT back to the webserver (which, on clamato.net, is not configured to accept PUTs, so that will always error). You end up with .st files you can manage in git or anything else.

You should be able to just ignore the commit errors and keep going through the tutorial, I think? But to get rid of them, just search for the "save" method, and remove the line that says "self commit", and then save it.

The script hangs and freezes in my browser, even on the tutorial page. I get the wait/stop script prompt every time I reload it. I'm using Firefox 3/Ubuntu.

Good to know - it definitely works best in Safari and Chrome right now (since they have the best-optimized JS implementations). I'm surprised Firefox is hanging that badly but I'll look into it.

The search filter seems a bit too eager. Type one letter and it tries to enumerate scrillions of methods, which kills Firefox.

It's probably the DOM manipulation specifically. Are you adding the results in one big innerHTML assignment?

You can also try breaking the search into small chunks with setTimeouts between them.

The problem is actually, more or less, building up the search index - this is only an issue the first time you do a particular kind of search, and it's fast after that. This involves parsing all the source code, which is a recursive process, which FF doesn't optimize (whereas Safari and Chrome zip through it much faster).

Neat. Is there a class hierarchy browser?

Reply on news.ycombinator.com