Hacker News Reader

Hacker News

Comment by rman666 | Hacker News Reader

Parent Full thread

rman666

It's not so much a question of "Why Lisp?" as it is a question of "Which Lisp?," if you ask me.

View on HN

Today you have 3 main alternatives:

* Common Lisp (used by the author of the article)

* Scheme

* Clojure

Scheme is not a single alternative. Racket [0] is a Scheme, which can be compared to things like Python or Ruby. There are other implementations, which should be used for different uses. For example, Chibi Scheme [1] is a nice alternative to Lua, but not to Python or Ruby.

Given your other two Lisps, I rather make this list:

* Common Lisp (which implementation?)

* Racket

* Clojure

[0] http://racket-lang.org/ [1] http://synthcode.com/wiki/chibi-scheme

I think Chicken[1] is a very attractive Scheme alternative. It's full-featured, fast, compiles down to C, has a very nice library (egg) system, and a relatively large and active user community (as far as Schemes go).

[1] - http://call-cc.org/

Hear, hear; I've been using Chicken for commercial and personal projects for about five years now. It keeps getting better.

Occasionally, Racket has a package that I'd like to port (e.g. datalog); but I find it otherwise overbearing.

Chicken has found some local maximum of efficiency and availability of useful libraries.

I would add Tiny Scheme as an embedding scripting language or as base of a DSL

One thing that intrigues me about clojure is that it affords some unique benefits, by virtue of being a lisp. For fun I want to try to write a clojure program which generates C code for a tiny game engine. e.g. you run the program, and it spits out .c and .h files which you can then compile and run.

Then, the challenge is to make it generate java code, too. I.e. in addition to the C code, which you can build and run, it also generates that same functionality in Java code, which you can run on JRE. So the game engine itself would be specified in clojure, and yet it would be "automatically implemented" across two very-different platforms! It's a fun challenge, I think. Maybe not doable, or doable only in a horrible way, but still fun.

But the point is this: that's a great example of a problem which would be pretty much impossible in a non-lisp language. In order to achieve that in Python, you'd probably be forced to implement a lightweight lisp within Python.

So that may demonstrate an answer to the question, "Why Lisp?" ... certain problems can be solved only in lisp, or by reimplementing the list-processing concepts of lisp.

There is already a project that translates ClojureScript to Scheme https://github.com/takeoutweight/clojure-scheme from which you can generate C code. Theoretically you could use the generated C code on any OS with a standard C compiler.

Why generate Java code if Clojure runs on the JVM?

Why is it "pretty much impossible" generate the game engine C code in a non-lisp language? What is it about Lisp that makes it possible?

The motivation behind "generating Java" was that the game could then be deployed in a browser, just like Minecraft is. Maybe Clojure can already do that though. I haven't looked yet. =)

Clojure runs on the JVM, but perhaps Clojure is expensive in the context of a 60-frame-per-second realtime app. I doubt it would be significantly more expensive than Java, but that's something to test.

So by generating Java, then it would be an interesting system because it winds up producing the same sort of code which Notch made by hand to deploy Minecraft to a browser. Yet it's derived "from the clojure code".

And yet it's also generating C code, which is also derived "from the clojure code". So it's very meta, which I can't resist exploring.

Why is it "pretty much impossible" generate the game engine C code in a non-lisp language?

Well, it's straightforward for you write a program which generates some C code. But how about "either C, or Java"? Or "either C, or Java, or Haskell", ... etc.

You'll soon discover that the nature of the implementation is completely different for different languages. A Haskell game engine would look utterly nothing like a C implementation would look.

So the situation is, you would need an intermediate source code representation of your game engine which supports introspection. And whatever data structure you use must also be able to be modified during the "code generation", because it's not until the codegen step that you know what sort of special-casing you'll have to do (in order to generate code which implements your game engine in e.g. haskell, or brainfuck, or... etc. Point being: each of those has a different set of special-case requirements, and you can't know which until you're already in the codegen phase.)

So, you could use a different language, and you could represent your "game engine source code" as, say, JSON. And you could have conventions for what your various operations "within that JSON string" actually mean -- e.g. whether you use infix notation, or prefix notation, or.. etc.

And then you could have a notation for how to specify a function, within that JSON string.

And so on.

But then, at the end of it all, you wind up with a poorly implemented version of what lisp already offers you. It lets you do all of that automatically, because those features are built right in to the language. Indeed, the language is the primary reason the features are possible: Everything is a list, and everything has access to all lists at all phases of the program's lifespan. You can run at compile time, or compile at run time, etc. You have access to the entire syntax tree at all times, in every program you write, which enables you to manipulate it. You simply can't do that with e.g. Python. Hence the requirement for an "intermediate thing, which you can manipulate" -- we happened to decide on using JSON as the representation, but you could've easily decided to use something else, etc. The point is, that "intermediate representation" is just a poorly-reimplemented version of the toolset which Lisp gives you by default.

I wonder if Python's metaclasses would get you what you're describing. Some thought would have to go into the primitives required to generate the constructs for the destination language(s), but metaclasses would allow you to serialize to different languages fairly easily once the primitives are in place. This approach isn't nearly as elegant as Lisp, but I think it takes what you're describing out of the "pretty much impossible" realm.

Overall though I agree with you, I just had a slight disagreement with the "pretty much impossible" claim.

I just had a slight disagreement with the "pretty much impossible" claim.

Thanks for pointing that out. Being a scientist, I too would take issue with this :) hence, I'm extremely interested in discovering whether this would be a workable, real-world solition for the problem I described.

I'm out of time right now, but would you mind emailing me? (Address is on my profile page.) I'd really love to get your thoughts about a couple followup questions that I have.

Thanks again!

And, It's really hard to choose one! All of 3 dialects has own unique benefits and lacks.

What does Clojure lacks in the language?

It's not the language that has problems, it's the environment. If you're already living in Java land, being tied to the JVM is a big plus. If you're not immersed in Java, you need to learn Java (and the libraries) and Clojure at the same time.

A similar point - if you want to interface with something written in C (i.e. anything not written in Java), it's going to be a lot easier in CL than in Clojure.

You don't need to know anything about java to learn and use Clojure.

then you have an error.

    user=> (+ "a" 1)
    ClassCastException java.lang.String cannot be cast to java.lang.Number
    clojure.lang.Numbers.add (Numbers.java:126)

then you look at a backtrace...

You still don't need to know anything about java to figure that one out.

I realize it's a contrived example, and I'm sure there exists an example where it would be critical to understand java to figure out what the deal is, but I haven't found it yet and I've been mucking in clojure for at least a year now.

I haven't worked with Java in a decade and it still bothers me that it has a NullPointerException when you can't actually manipulate pointers.

Clojure adds syntax, which breaks some of the inherent symmetry of Lisp slightly[0]. This isn't something you're likely to notice unless you do a lot of heavy metaprogramming.

Clojure cannot guarantee elimination of tail calls, because of the limitations of the underlying JVM. This also means that it can't properly handle corecursion in the same way that pretty much every single other Lisp can.

(Both of these have been discussed before many times before on HN, so I can pre-empt the next comment in this thread, which will be someone pointing out that Clojure provides "loop" and the "recur" macro - to which I'd respond, yes, they're logically equivalent in the end, but having to force the transformation to a loop explicitly breaks the paradigm, which for me is the whole point of using a Lisp. This is one of those topics that can be discussed ad nauseum with no "conclusion", so it's not worth spending too much time on it.)

Lastly, anytime you're dealing with binary compatibility between various JVM languages, the abstraction is inherently a bit leaky. I haven't used Clojure myself, so I can't comment specifically there, but from my experience with using Java libraries in Scala, I can testify that some of the warts of Java end up leaking into Scala code. Nothing debilitating, just a bit frustrating[1].

[0] Racket does too, but the "syntax" added (square brackets) has the same semantics, so it's really just an equivalent token - an alias, if you will.

[1] One particular example I remember has to do with how Foo.class in Java works, and how a library dependent on this particular pattern has to be used in Scala - it just gets a bit messy.

> Clojure adds syntax, which breaks some of the inherent symmetry of Lisp slightly[0].

In what way does this break the symmetry?

Lisp source code is a Lisp data structure (or becomes one when read by the reader). In Common Lisp and Scheme, that structure is either an atom (an integer, a string, a symbol, etc.), or a list consisting of cons cells. In Clojure, that structure can also be a vector or a map. This is enabled by the fact that vectors and maps have their own literal syntax.

I was uneasy about this in the beginning as well, but then I came to the conclusion that this is not unlispy at all.

Symmetry may not have been the correct word, since it does provide referential transparency but it certainly does add an extra layer of complexity to the parsing, even if it's only one extra step. Furthermore, I would argue that the additional syntax isn't necessary, which is my biggest beef with it - since you can convey the semantics of a vector or a map without altering the syntax at all, there's no reason to complicate the syntax any more than needed.

> In Common Lisp and Scheme, that structure is either an atom (an integer, a string, a symbol, etc.), or a list consisting of cons cells.

Let me fix that for you: in other Lisps, an item is either an atom or a cons cell. There's "no such thing" as a list in Lisp.

There's a huge difference between having an option with two outcomes (S -> atom | cons) and an option with three outcomes. In computer science, we count "zero, one, many" - booleans are an example of this. By adding a third option, we've stepped out of the realm of the binary into the "many", and that's a much messier world to deal with.

> I would argue that the additional syntax isn't necessary, which is my biggest beef with it - since you can convey the semantics of a vector or a map without altering the syntax at all, there's no reason to complicate the syntax any more than needed.

You could say the same about the quote, backquote, unquote and unquote-splicing syntactic sugar being built into the reader. It is redundant, and yet it's there in most Lisps -- because it helps readability/maintainability at the cost of the little complexity it adds.

> Let me fix that for you: in other Lisps, an item is either an atom or a cons cell.

In Common Lisp, it is only correct insofar as the language defines "atom" as "not a cons cell" [1], contrary to the intuitive understanding that it's an indivisible entity. E.g., CL vectors are atoms, even though they have more in common with lists than, say, symbols. And they do have literal syntax, like #(1 2 3). How is that different from Clojure's [1 2 3], save the different type of parens?

[1]: http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec...

This has been in Lisp since the big flood.

Common Lisp:

    * #(1 2 3)

    #(1 2 3)

    * (type-of #(1 2 3))

    (SIMPLE-VECTOR 3)

Being a bit of a pedant: do you mean "mutual recursion" instead of "corecursion"? The former means multiple functions calling each other, thus being recursive indirectly. The latter is more like an inverse of recursion: instead of starting with some data and reducing to base cases, you start with base cases and produce (co)data.

Defining a lazy list in terms of itself is an example of corecursion, which Clojure can do (albeit a bit awkwardly because you have to make the laziness explicit).

I wonder if, in the future, this limitation of Clojure will go away. I know there has been talk of the JVM folk adding things like support for tail recursion. That said, I'm not smart enough to know whether talking about adding support for tail recursion translates to "Once we're agreed, let us go ahead and solve this tractable problem," or if it translates to "Gee this would be nice, but it's wicked hard."

> support for tail recursion

Let's be clear - it "supports" tail recursion; it just uses more than a constant amount of space on the stack to handle the calls - in other words, it doesn't transform the recursive call to a loop.

Eliminating tail calls is by no means a hard problem in the general case - it's on the order of something you might be expected to do in an introductory compilers class.

The problem is particular to the JVM. As I understand it, is due to the fact that the JVM was architected in a way that doesn't allow it to guarantee that a tail call can be transformed into a loop. These techniques certainly existed in the early 90s (they've existed since, what, the 60s?), but at the time, I guess people didn't think it was a priority.

I have no further knowledge of the JVM, so I can't really comment on whether or not they'll be able (or willing) to add the support, but the problem is dealing with legacy designs/systems, not a difficult problem of CS theory.

> Eliminating tail calls is by no means a hard problem in the general case - it's on the order of something you might be expected to do in an introductory compilers class.

Actually it is problematic. Especially the interaction with dynamically scoped constructs...

It's depending on project/person. For me it's JVM (but of course JVM is big benefit also), mostly because with JVM I'm not able to create library which might be used in native code for example in iOs app (theoretical it's possible with ECL,Clozure CL or "Scheme to C" compiler).

Common Lisp. Everything else is so much Lisp as php is perl or C++ is C.

Reply on news.ycombinator.com