Lispsyntax.jl: A Clojure-like Lisp syntax for julia | Hacker News Reader

Lispsyntax.jl: A Clojure-like Lisp syntax for julia

github.com

162 pointsphonebucket52 comments

Great example of building something fun not because we should but because we can :)

My take on "Lisp in Julia" using deprecated-but-still-parseable syntax instead of a string macro: https://github.com/christopher-dG/jlisp

I should probably put some examples in the readme.

That hidden Julia s-expression syntax helped me start to understand metaprogramming in Julia. Along the way I found that it takes surprisingly few lines of code to turn the normal Julia repl into a more-or-less working Julia-lisp repl: https://gist.github.com/brenhinkeller/44051118c2f9d18b26dc76...

While not nearly as mature as the main platforms (like the JVM, .NET, BEAM), the Julia compiler does have some unique features (an extremely aggressive optimizer, a sophisticated type inference mechanism, possibly the most exceptional multiple dispatch support, Go-like lightweight processes), including a macro system that supports reader macros for writing parsers (like this library) and extensible multi-stage compilation (like the IR manipulation that is used in auto-differentiation of source code), which makes me wonder if once all those features (and possibly new ones) become robust enough it will become one of the targets for people creating new programming languages.

What you see most is domain-specific languages implemented through macro's. For instance this package, https://jump.dev/JuMP.jl/dev/examples/basic/, https://github.com/SciML/ModelingToolkit.jl or https://turing.ml/dev/docs/using-turing/get-started

Yes, I was thinking about someone going many step furthers, like Elixir is for Erlang and the BEAM, someone who disagrees with design decisions of the Julia language but appreciates the compiler's capabilities and the ecosystem enough to use them to implement their own vision for a programming language. Making it a Lisp is just one of the more obvious scenarios, as someone may like Julia in general but want the full power of s-expressions.

How is Elixir’s design different from Erlang’s? Semantically they’re pretty much the same language.

A better example would be Java/Clojure.

Elixir is basically Erlang, just like Lisp Flavored Erlang and Gleam (the statically-checked language for the BEAM). The BEAM in general is a very specialized VM, which is why I used as an example, as the Julia compiler is also a very specialized platform and other languages targeting it would also be just Julia. Like Julia with s-expression, a typescript like extension for Julia that maybe would focus on different tradeoffs by reducing polymorphism in exchange for type checked contracts (but still use the same machinery and ecosystem since it would compile to Julia/Julia IR). I really wouldn't expect something completely different like the JVM supports (as at this point it would be better to use the LLVM directly).

Most languages designed to be used by people are not good targets for people creating new languages. Very different set of concerns.

The main ones I am aware of are:

JavaScript: you basically got to if you want to run in a browser.

And C Because it's available on every piece of hardware, it was there before llvm and it doesn't change across hardware (unlike assembly), and most people writing languages know C.

JVM, the .Net CLR etc are not languages. They are intermidate representations like LLVM is. (Approximately). They were designed to be compilation targets. Julia wasn't.

Most of Julia's intersting compiler features are unusual because they need to solve problems Julia has in general other languages wouldn't,or that would not transfer across language boundaries.

The macro stuff is awesome for DSLs. Messing with compiler passed is cool for adding capacities. But I think much more in the sense of implementing a micro language that is closely related to Julia, rather than a full on language that has its own goals and identity (I want to write one for codegolf called Jules that is just Julia except on an error it tries some other lexically nearby code)

A fun Julia easter egg I recently discovered.

Running 'julia --lisp' launches a femtolisp (https://github.com/JeffBezanson/femtolisp) interpreter.

The Julia parser is actually written in femtolisp: https://docs.julialang.org/en/v1/devdocs/eval/#dev-parsing

https://github.com/JuliaLang/julia/blob/master/src/julia-par...

My current favourite of these is this Lua version: https://github.com/bakpakin/Fennel

> Optional typing - Currently not implemented

Jesus H Christ could the whole FP world stop this? You're either going to prioritize static types or you're going to blow them off. Either front-and-center your type system, or stop mentioning how you blew it off. Those of us who want Clojure's syntax and semantics and a decent type system would like to stop being taunted.

I think you may have misunderstood what you're looking at. Julia is very much a dynamic language. It also has the ability to add type hints, but, unlike for most optionally typed languages, these aren't just type hints for human readers or an external type checker. They're actually one of Julia's more interesting features. Julia's type declarations provide information that the compiler uses to generate faster code, and to support its multiple dispatch functionality.

When the notes say "optional typing - currently not implemented", all they're saying is that this syntax doesn't allow you to tap into that feature, because a syntax for type declarations has not been provided.

Just a minor point - type hints only really help if you put them in struct fields. In function signatures they only limit what can be dispatched to that specific method, there's (almost always) no difference to leaving it untyped since functions are compiled for specialized types anyway.

I just remembered of another one, seem to still be actively worked on: https://github.com/LuxLang/lux

I’m kind of confused by your comment. What is it that you’re unhappy about?

Basically every new lisp that comes out has a big bullet point of features and included in the list of features is a link in the README about static type checking. They all invariably raise the hope of a statically typed lisp for the given use case, then clicking through to the linked heading say “this project isn’t focusing on static types”. It’s Lucy and Charlie with the football. If you’re developing a lisp and you’re not interested in a static type system, not mentioning it is good enough. Making it a big headline only to yank it away in the rest of the text is just a disappointing setup.

I think you’re confused about this project. This is not a new language. It’s literally just lisp syntax for the Julia language.

When this gets typing, it will be with Julia’s very powerful and expressive type system (which is dynamic by the way), and it just needs someone to modify the LispSyatax.jl parser to properly represent the type annotations.

Calm down.

> I think you’re confused about this project

You’re the second person to say so, and it’s entirely possible I’ve misunderstood the language I quoted that’s nearly verbatim in almost every new lisp that addresses static types at all.

> When this gets typing

I’m not entirely clear that there’s any intention of it getting typing. Am I missing some statement that “not implemented” is temporary?

> Calm down.

I’m sorry I took my exasperation out on this project which I’m sure is awesome. It’s just a very general frustration I feel about the priorities of the FP+lisp community generally. Like I expressed, it’s exceptionally common to announce a new lisp (language or syntax) where static types are explicitly called out as a headline non-feature.

It’s disappointing because every time I see “lisp”, especially “Clojure like”, and I see types addressed explicitly... I keep hoping someone’s come along and married a syntax and (hopefully) state management approach I adore with a static analysis DX I also adore. And 100% of the time so far it’s been... “Types? Glad I got your attention, this isn’t for you!”

I’m well aware I’m not entitled to other people building the language I want. I’m even well aware I could build it myself, probably on top of these existing lisps.

I’m just disappointed to see so many projects explicitly identify something I want in a way that feels hopeful and bury the “nope we didn’t actually bring a type system to something like clojure” behind the headline.

I’m not losing sleep over it (other than to type this between falling asleep on the couch and proper bedtime), but can you understand how that messaging is a disappointing thing?

There's a strong tradition of dynamic fp-ish languages with Erlang, Scheme, Common Lisp[1], and lately Clojure (along with many others) that just isn't about static typing. A lot of people feel that dynamic + fp is just the right thing. It's not an omission, since putting in static typing restricts, complicates and remolds a language a lot.

[1] Yes, there's optional type declarations in CL, as an add-on feature. But I'd guess most CL programs don't elect to use them.

I’m well aware that most lisps and BEAM languages are dynamically typed. My objection wasn’t that they didn’t include static types, but that there’s also a strong tradition of these languages including a prominent headline about static types, which gets my hopes up that this one is bucking that trend, only to have my hopes dashed. It’s so much a general expectation that a lisp will have a dynamic type system that it’s safe to assume and could go without saying.

TXR Lisp doesn't mention static types:

  $ grep 'static type' txr-web/*.html txr/txr.1
  $

No hit for "static type" in the web pages or reference manual. Dynamic type ditto:

  $ grep 'dynamic type' txr-web/*.html txr/txr.1
  $

I don't think Common Lisp has a "prominent headline" about static types anywhere, either. There are no hits for any of those terms in the draft ANSI standard:

https://franz.com/support/documentation/cl-ansi-standard-dra...

Scheme's R7RS explicitly talks about dynamic typing in two places:

Introduction:

Scheme was one of the first programming languages to incorporate first-class procedures as in the lambda calculus, thereby proving the usefulness of static scope rules and block structure in a dynamically typed language.

1.1 Semantics:

Scheme is a dynamically typed language. Types are associated with values (also called objects) rather than with variables. Statically typed languages, by contrast, associate types with variables and expressions as well as with values.

That's also the only mention of "static type" or "statically typed".

> It’s so much a general expectation that a lisp will have a dynamic type system that it’s safe to assume and could go without saying.

A language reference manual has to document the type system so that someone knowing nothing about language type systems can understand it. This can be done without using dynamic type terminology or mentioning static type checking, but it has to be done.

There is a lot of detail there; no two dynamic type systems are exactly alike.

Again, this is not a separate language. This is just a different surface syntax for julia programs. The type system already exists and is implemented.

The only thing that needs implementing is a syntactic form.

The "Optional typing - Currently not implemented" not-so-big bullet point is under the big "Notable Differences" heading.

If it were not mentioned, the implication would by that it was available.

This is the closest I've seen: https://github.com/carp-lang/Carp

Sorry for double response but I took a glance and have some stars in my eyes! I’m not sure this is the language I imagine but it’s awesome awesome that it exists and someone is working on something even a bit like a Clojure with a real static type system I would want to work with.

Thank you for mentioning it! I’m gonna take a look tomorrow!

What is the magic behind Lisp anyways?

Is it that the Lisp language exposes the Abstract Syntax Tree (AST) of the code, so that you can reformulate it into your domain-specific macros, thereby helping you encapsulate complex logic into more succinct constructs?

Is this why they say that Lisp is fantastic for small groups of programmers, that share similar groupthink, and uses all the same macros, but it becomes a horrendous mess when multiple teams are involved.

> magic

I'd say yes, this, though you don't actually manipulate a proper AST, but a list (tree) of program instructions. An AST would be given by a "code walker". Plus, the syntax is small and coherent, the language is stable, the syntax makes it straightforward to add new language constructs that would need a language release for another classical language. With most implementations of Common Lisp, you code against a live image, so you get instant feedback: compile a function with a keybinding, see compiler warnings or errors instantly, try it right away in the REPL (no process had to restart), get an interactive debugger on an error, fix it and resume the execution from a chosen frame (no stack unbinding), inspect objects, change a class definition and have existing objects being (lazely) updated, given rules in the standard that you can control… when ready, build a binary, and deploy. Today, SBCL's compile-time type-inference warnings are very helpful.

Of course, some companies use CL in a million-sized codebase (if that helps as a counter example): Google (ITA software), SISCOG (underground and rail transport optimisation), ACL2 (industry-strength theorem prover)…

https://lisp-lang.org/success/

https://github.com/azzamsa/awesome-lisp-companies (disclaimer: these resources are not complete)

This sounds incredible. I wonder why Lisp didn’t catch on as a more popular programming language.

I can give you some pointers, as a relatively young lisp developer (came to it ±3 years ago): at the time, unappealing official website (now fixed); only books to learn, few online code-first resources, so was harder to learn than necessary (greatly improved); only Emacs & Vim first-class support (now we have very good Atom and VSCode plugins, and more (Jupyter kernel,…)); hard to find libraries (only Cliki existed, it's now better, and there are plenty of libraries); Quicklisp seemed limited from the outside (although it's great) (we now have Ultralisp and the CLPM package manager, along with Qlot (directory-local installations), utilities like Roswell etc); hard to know if the language was being used at all (we now have lisp-lang.org showing success stories, awesome-lisp-companies shows more examples); hard to get into web development / no major framework (the Cookbook helps and there are many choices, although still no world-class framework. But I use a Lisp backend, classical Django-like templates and it's enough to build real-world, work-related stuff); no good GUI options (there was Qt4 without much examples, and Ltk. We now have IUP bindings, some more, and a Qt5 is coming soon©);…

Others know better the limitations of Lisp systems of the 80s or 90s. The compilers also improved (SBCL; there is CLASP in development to interface with C++ code, and more)

TLDR; it's catching up :D

> Is this why they say

No, why they say is because they have zero experience with Lisp, in any size project.

And there exists no tool using which some people have not made big, huge mess. We can easily lob this criticism at anything, quite randomly.

Imagine a tool which has the property that a team of doofuses, no matter how large (in head count and doofiness), cannot make any sort of "horrendous mess". What sort of limitations does that entail, and could any of us here live with them?

So what has your industry experience with Lisp been like?

lisp program is the program, not a representation of one like in all other kinds of languages (except machine code, which isn’t really a language).

The irony here is particularly rich considering Julia's parser frontend is written in Scheme.

It seems more fitting than ironic to me. Julia's never been shy about its lisp heritage, and the reason why this project is only a couple hundred lines of code is that Julia remains very close to those roots.

Certainly any alternative syntax parser for any language would be?

It would also only take a couple of hundred lines to write a parser for a C compiler that would allow that.

Not any to any. You could't write a thin lispy alternative syntax for C because of some semantic differences. "If" is an expression in Julia and Lisp, and it's a statement in C. Dealing with things like goto statements and preprocessor macros would also require some extra work.

Add all that up, and you're no longer looking at alternative syntax, you're looking at a transpiler.

But this is also simply an alternative Syntax that mirrors Clojure, not actual Clojure semantics.

I think we're talking about things different ways around. I'm talking about doing something like what's in the link: an alternative syntax for Julia that mirrors Clojure but retains Julia's features.

You would be harder-pressed to create an alternative parenthesis syntax for C, because C is so syntactically complex.

While julia does not have full lisp semantics, it is very close. Granted, it's not especially similar to Clojure itself. It's closer to a lisp though than almost any other relatively popular modern language that doesn't use S-expressions.

I'm very sceptical of such claims because I do not believe there is such a thing as “Lisp semantics” and that it is primarily a language of syntax. In particular, Common Lisp has many semantic features such as dynamic variables and the function–value dichotomy that do not exist in, say Scheme, or really any other commonly used programming language. And Scheme, yet again, has many rather esoteric operations, such as continuation capturing, that exists nowhere else as well.

The three most popular Lisps today are C.L., Clojure, and Scheme, and I don't see these three having unified semantics at all and they are often further from each other than they are to many other languages.

There are julia implementations of continuation capturing [1] fwiw. Broadly though, I agree that 'lisp semantics' is a very fuzzy term. I'd claim this family has several strong semantic themes despite their differences. E.g. their approach to meta programming, their approach to dynamism, lexically scoped closures, and the everything-is-a-expression rule. Julia fits all these themes (and others), though it's dynasmism is a bit more restricted than most lispy languages, and it's AST is printed differently, even if it's internally represented and manipulated the same as other lispy languages.

You're probably right though that we wouldn't be tempted to group them together nowadays if it weren't for their syntactic similarities.

[1] https://github.com/MikeInnes/Continuations.jl

Virtually every modern language has lexical closures; dynamic typing is certainly not unique; and not everything is an expression that returns values at all.

That last point is just wrong. Sometimes the return value is `nothing`, but that's still very much a return value.

A couple years into fatherhood and I see puns everywhere, and suspect others of dropping them too.

Particularly Rich... Hickey?

I juxt ignore the puns.

Where did you steele that joke from?

Why is that ironic?

I'm not sure, but maybe they're unhappy that it uses a more Clojure style syntax?

At the very least it is amusing.

How about a Julia-like syntax for Clojure?

Reply on news.ycombinator.com