Introducing lisp.js: a Lisp for node.js | Hacker News Reader

Introducing lisp.js: a Lisp for node.js

lisp-js.posterous.com

64 pointsdevijvers30 comments

While it's always great to see experimentation in the Lisp realm, I'm unclear as to what "a Lisp for node.js" will do that the mature Lisp-on-top-of-JS (Parenscript) doesn't already do.

There's also already a Lisp-on-Node project called Sibilant. Jake's got a neat live-compile-in-the-browser demo up here:

http://sibilantjs.info

And the source is available here (and as an NPM package):

http://github.com/jbr/sibilant

I went through the demo and have the same question about this: why greenspun Parenscript in this manner? It does all these things, and zillions more, in a nice way. The only reason I can think of is fun, which is perfectly legitimate. Or maybe people don't want to run Common Lisp even to compile their code?

As the author of sibilant: A) it was/is fun B) I don't want to be dependent on another language — sibilant is entirely self-hosting, like coffeescript. Additionally, I don't want to be constrained to pure lisp. I'd like for sibilant to evolve into a language that incorporates the best elements of lisp and javascript. Since sibilant is written in sibilant, there's no dependence on pure lisp and the language can evolve in its own direction.

There's something satisfyingly elegant about using an interpreted language to interpret another language - macros can actually modify the compiler at compile-time, from within the language. I hope to make more use of this possibility in the future, as it's currently only used for macros.

Some reasons I can think of: it's definitely an advantage to be able to compile the code on the client-side, which sibilant should be able to do; I don't know if Parenscript can bootstrap itself the same way. Some people, like me, prefer lisp-1 to lisp-2. Being less like Common Lisp and more like Javascript can be an advantage; sibilant seems to support foo.bar notation for accessing JS properties, which I think is the right choice in this context (I would also like to see JS/JSON literal syntax supported).

All of the docs on Parenscript seem to assume you're using it to render bits of JS and HTML from a Common Lisp server. It would be useful if there were a tutorial that showed how to use Parenscript for this use case, simply to compile lisp-like code into standalone JS files for use with whatever arbitrary backend.

Yeah, perhaps I'll write something that once we have a bit more experience with server-side PS.

Interesting what you say about foo.bar; PS originally did that, and I was one of the people who lobbied against it. It does make getting started a little easier, but interferes with macros down the road. For example, if you want to do anything like transforming foo.bar to foox.barx, you end up having to parse the symbols to split them, which seems wrong. Since Lisp-style metaprogramming is pretty much Parenscript's raison d'être (well, that and interoperability with CL for the crazy few who care), it seems foolish to do anything to compromise it.

Why not consider it a reader macro, and translate it to (@ foo bar) or whatever is appropriate, before applying macro expansion?

I considered doing that, but it pretty much means rewriting the reader because it changes the grammar so much. Not that it's not possible, I just don't trust myself not to make it full of bugs.

Seems possible. You'd have to make . be a macro character, which would ruin dotted lists, but Parenscript doesn't have those anyway.

I don't know this particular author's intentions, but I can definitely see a reason to create a new Lisp over Javascript: I like functional programming, so I'd like something Schemey or Clojurey rather than CLish.

Thanks for the reference to Parenscript which I'm unfamiliar with. The main point of lisp.js is to benefit from node.js' event-loop/non-blocking-i/o environment. node.js will be able to handle far greater loads than web servers that use blocking I/O. I'm not sure whether or not the Common Lisp web server exclusively uses non-blocking I/O on the account that I'm not familiar with that web server. I'm erring on the side of blocking I/O in which case node.js/lisp.js would run circles around that web server.

Using Parenscript with Node.js would presumably mean throwing out any Common Lisp web server and relying on Node to do the I/O. Otherwise, as you point out, you don't get much benefit from Node. In such a setup, Node (or Node behind a reverse proxy) would be the web server, Parenscript code compiled to JS would do whatever was needed in Node, and if you wanted application logic in a Lisp at runtime, you could call it however Node does IPC.

The value that Parenscript adds here is the same as for programs that run in a web browser: you write your JS in a Lispy way at a higher level of abstraction, mostly because you get the full power of Lisp macros. There are other wins too, but that's the big one. The runtime environment, though, is entirely JS, because PS is a compiler that targets JS.

So I still don't see a difference in intent here between what PS is good for and what you're doing. I don't want to discourage you at all. Just curious about the differentiator.

I've scanned the Parenscript docs and it doesn't seem immediately obvious how to combine node.js and Parenscript without having to include Common Lisp. Also, there's a number of things on my wish list that compelled me to write my own lisp implementation (there are other JavaScript lisp implementations):

* only requirement is node.js and the lisp source distribution, no other dependencies. * ability to debug lisp code from the browser. * immutable data structures.

I feel I can implement a lisp that can run the examples in <i>Let Over Lambda</i> in a couple of weeks. The lisp won't be as stable as Parenscript but that's ok. Only at that point can the real work start of building the web framework that is lisp.js. I think the most important bit for me is to have a base that I can modify as the needs arise. I'm already trying to include as many lisp concepts in the test source as I can find to make sure lisp.js is a general purpose lisp. In that respect I'm setting several goals: get lisp-in-lisp working, get Let Over Lambda examples working, ... any other useful lisp constructs that I come across. That's a journey I can only make if I do my own lisp implementation.

Regarding using NodeJS with ParenScript: it is really easy.

I am using Node on the backend of http://tryparenscript.com/ simply because it was easier to get up and running than Hunchentoot. Source code is on GitHub, if you want to check it out: http://github.com/fitzgen/tryparenscript.com

Here is the "Hello, WOrld!" example from the NodeJS homepage in ParenScript:

    (defvar *http* (require "http"))
    (chain *http*
           (create-server (lambda (req res)
                            ((@ res write-head) 200
                                                (create "Content-Type"
                                                        "text/plain"))
                            ((@ res end) "Hello, World!\n")))
           (listen 8124 "127.0.0.1"))
    ((@ console log) "Server running at http://127.0.0.1:8124/")

Far be it from me to stand between a programmer and his own Lisp implementation.

I'd love your help on sibilant; I started it with exactly the same motivation.

I like this! I just put a reminder on my calendar for 3 months from now to more-fully check out the project.

Awesome. I've got it up and running. I've been looking for a solution for the client/server distinction for a while. How much have you thought about the implementation of that?

Good to hear! What do have in mind when you say "client/server distinction"?

Mainly, I'm tired of using $.getJson, and I liked your on-server macro. Also, how is the server going to watch references on the server and vice-versa?

Basically, the server "knows" (as in can know) the references that exist on the client and vice versa. When a watched reference changes value on the client an HTTP request/response is sent over the wire. The request carries the old and new value, the response is basically redundant but since it's HTTP it needs to be there. Hence, the response might carry new forms to be executed on the client.

The client watching references on the server is obviously more tricky. node.js already supports keep-alive by default. There probably needs to be some kind of chuncked reply mechanism that allows the server to send changes without the client having to send polling requests. Obviously, if the reference is not going to change frequently it might be cheaper to just poll every once in a while.

on-server reminds me of Hop. Take a look at http://hop.inria.fr/, maybe http://hop.inria.fr/usr/local/lib/hop/2.1.1/weblets/home/art... in particular.

lisp.js will be a lisp-2 (like Common Lisp and unlike Clojure and Scheme)

Can you talk more about this decision?

Macros are important, that's why I seeking guidance in Let Over Lambda. The most suitable lisps for writing macros are lisp-2, hence the preference for a lisp-2 dual namespace.

Lisp-2's have less potential for trouble with unhygienic macros, but they still have it. Hygienic macro systems work with Lisp-1 as well as with Lisp-2.

Apropos, I've written a Lisp->JS compiler a while ago; you might find some of the code useful: http://github.com/manuel/cyberlisp/

In regards to hygienic macros and Hoyte's Let Over Lambda, the type of variable capture hygiene is designed to prevent is actually used as a programming technique in the book; Hoyte calls it free variable injection.

Lisp-1 vs Lisp-2 actually has very little to do with macros. Most of the inconvenience comes when writing functions - things like having to name list parameters "lst." The way Common Lisp gets around the variable capture problem is to forbid the rebinding of special forms, functions, and macros defined in the standard.* This is an ugly hack that forces early binding, and obviously does nothing to prevent capture of user-defined functions.

The Scheme wiki has a good article on hygiene: http://community.schemewiki.org/?hygiene-versus-gensym

* - This wasn't done for the sake of macros. The rationale behind the decision as I heard it is to prevent the hypothetical scenario of some standard function like cdr that's called inside the garbage collector or allocator from being rebound to something that tries to allocate memory and crashes the gc or sends the allocator into an infinite loop. I don't really buy it; it's easy enough to prevent these types of scenarios.

You might also check out the Racket->JS compiler:

http://planet.plt-scheme.org/display.ss?package=moby.plt&...

I used it just last year — the developer, dyoo, is super responsive.

I'm sure he'd be happy to talk shop with you: http://github.com/dyoo/moby-scheme

I think the Scheme and Racket communities would take issue with your claim that lisp-2 languages are more suitable for writing macros than their respective languages :)

So you mean "Lisp for JavaScript"? Why limit it to node.js?

It might one day become a general lisp for javascript but for now the concurrency-free server-side environment of node.js is assumed.

Reply on news.ycombinator.com