Hy: A dialect of Lisp that's embedded in Python | Hacker News Reader

Hy: A dialect of Lisp that's embedded in Python

github.com

166 pointslnyan55 comments

I know a lot of people are going to poo poo on this as "not a lisp" but I think it makes lisps more approachable for a python programmer who is not used to them.

I used experimented with Hy a few years ago and I felt that it made it easier to write recursive functions (even if python doesn't have tail call optimization). Being able to bind multiple values to multiple variables was nice, and I believe I could get multi-line lambdas in Hy as well. And it wouldn't be great but if someone new Python by not Hy, there was always hy2py to convert your Hy program back to Python. It was also nice to be able to write some sexps but still have the entire python ecosystem.

Doing dataframe manipulation through parens with pandas made me laugh, and while it wasn't always clear how to translate my python code into Hy It was definitely waay easier then learning an entirely new language.

I eventually moved onto Racket after I ran into some issues with Hy being "not a lisp" but Hy certainly made Racket easier for me to grok after playing around with it

Side comment: Python doesn't have TCO, but Hy does have the loop/recur macros in its contrib library. They give you a version of tail recursion that looks a bit odd at first, but quite nice once you get used to it.

Hy's loop/recur had a serious bug last I checked: https://github.com/hylang/hy/issues/1465

But see the @loop decorator from Drython for how to do this in Python: https://github.com/gilch/drython/blob/eb1773c14060e31e2544f5...

This exists now, and it's awesome.

https://github.com/clj-python/libpython-clj

hahaha, even better! Man sad thing this isn't going through the roof yet!

Amazing!

I am currently writing an AI book using Clojure examples. I am using this library, as well as DL4J, etc. I am a Common Lisp developer, but I hope that my Clojure book will be useful.

I wrote a short Hy language book, mostly as a learning activity for myself. You can get a free copy on the books page of my personal web site [1].

[1] https://markwatson.com

EDIT: to get started with libpython-clj go to Carin Meier’s great example repo https://github.com/gigasquid/libpython-clj-examples Also, libpython-clj can be tricky setting up so I use a dedicated GCP VPS that I spin up just for using this library.

I'm looking forward to your Clojure AI book. Your books look amazing. Thanks for sharing.

Thanks! I hope to have the first edition out in 6 to 8 weeks. As always, there will be a free PDF download on my site.

Note that this is not Lisp interpreted in Python, but transpiled into Python AST on the fly. Which is how Lisps should be implemented on top of modern environments.

(IIRC the ‘gisp’ project takes that approach even further: it transpiles a Lisp to the AST, and then spits out Go code that would generate the same AST: https://github.com/jcla1/gisp)

The mapping of the AST onto lisp plus an implementation of macros would solve all the issues with Go verbosity, lack of generics, error handling, and probably much more.

But it would have a lot of braces so junior programmers wouldn't be able to use it and so it's a non starter.

You can translate an analogous syntax of your choosing into the same lists =) E.g. something Python-ish or Yaml-ish, or Rebol.

See Hebigo's Python-like approach: https://github.com/gilch/hissp#hebigo

This seems a bit like the story with Clojure vis-a-vis Java, to me. Yes, Clojure does a lot to clean up Java's verbosity, flaky implementation of generics, etc. But I don't think that's actually because Clojure is a Lisp; I think it's mostly because Clojure is a dynamic language. For example, Clojure largely resolves Java's generics situation by simply not needing them in the first place.

That worked for Java, where a lot of people are stuck on the JVM for legacy reasons, but yearning to at least be able to work in a more ergonomic language. Go lives in a different social environment, though. Nobody's stuck on Go because they've got 25 years' worth of massive legacy Go monoliths that they can't just chuck out the door. They're on Go because Go is what they want. And I'm pretty sure, given that they're invested in Go, that a dynamic language with a heavy focus on metaprogramming is approximately the opposite of what they want.

I keep meaning to play with https://github.com/cosmos72/gomacro

It's a Lisp-flavored Python, Python with Lisp syntax.

Just doing a syntax translation into an AST might not be enough to implement an actual Lisp on another language.

I spent some time with Hy about a year back, and that is approximately the impression I brought away. In order to maintain good interop with Python, they had to make a lot of compromises about how things work in Hy, relative to what a Lisper might expect. The most striking example I can think of offhand is that `let` had to get banished from the standard library: https://github.com/hylang/hy/issues/844

That said, Hy is still a nice language, and very well thought out. It's just that billing it as a lisp dialect for Python (as the project's website does) might lead to some false expectations. That GH thread I linked above is a great example of this. There's a lot of good, careful thought going into the design of the language. But it also has this sentence in the opening comment: "Hy is not Clojure, nor Common Lisp, but homoiconic Python." If you're interested in a Python variant with good macro system, this is it.

(Also, I'm not sure it necessarily disqualifies Hy from being a lisp. There are too many different conflicting opinions about the definition of 'lisp' for me to want to say either way, and, while a lot of good and insightful things have been said on the subject, I've personally always found that particular argument to be rather boring.)

Hy does have a let macro now, but you have to (require) it before it works: https://docs.hylang.org/en/master/api.html#module-hy.contrib...

> Which is how Lisps should be implemented on top of modern environments.

Lisps are modern environments. Which is why they're implemented in themselves.

Cool, I thought it was dead (like the fictional character called, coincidently, "Snake"). I see that active development has restarted 6 months ago, seemingly. Kudos to everyone involved, specially @Kodiologist who seems the main contributor over the recent period.

(Shameless plug: more functional languages that look like Python, or compile to one of the Python VMs: https://github.com/sfermigier/awesome-functional-python#lang... ).

You should add Hissp to this list: https://hissp.readthedocs.io

Karen Rustad Tölva drew the Hy logo, Cuddles the cuttlefish, and later the Rust mascot too, Ferris the Rustacean.

You can probably see they are in the same style :)

Paul Tagliamonte and the Hy contributors is what got me into F/OSS many years ago, and even if I haven't actively contributed the past 6-7 years I'll have a veryvery sweet spot for the project<3

I was involved in GNU MediaGoblin at the time (circa 2011), and both krustad and paultag spanned those communities. I have fond memories of the time.

I also remember your name!

There's a cool book about it here https://leanpub.com/hy-lisp-python

Is it possible to losslessly translate from Hy or Python to the Abstract Syntax Tree and back? That is, is there way to have a code viewer that, instead of highlighting syntax, changes it to another syntax on the fly?

Could each member of a development team look at the same piece of code but the first sees Lisp syntax, the second sees Python, a third sees a C-style syntax, and a fourth has a mishmash of their own design?

Obviously there are language differences that go beyond syntax, but the learning curve of reading unfamiliar syntax often seems to be a sticking point for people considering/learning a new language. For example, someone who has experience in curly-brace languages may initially have trouble adapting to the use of indentation in Python or parentheses in Lisp.

Lisp and Python have different semantics, even between Python and Hy: an obvious one is that Lisp's `if` and all other statements are expressions. Coming up with a syntax is not the hard part.

A couple gotchas that basically all Lisps-on-other-languages have to keep in mind are: variables, since Lisps usually don't have the same scopes as anything imperative (with the ‘let’ blocks); and function names and possibly functions-as-variables, since Lisps are liberal with both. As a rule, you'd have to implement your own variable scopes and name mangling for where the semantics don't match. Plus macros are an obvious addition—IIRC Hy still has them in separate files (though I might confuse that with Fennel). And if you wish to have keywords as a separate type, that's usually entirely on you—afaik Clojure went full-in here, others not so much.

> reading unfamiliar syntax often seems to be a sticking point for people considering/learning a new language

Precisely the opposite of my and many other people's experience. Remembering whether blocks are delimited with braces, indentation or e.g. ‘then/end’ is a no-brainer. Learning the subtle differences in semantics, the type system, the library, package system and build environments—that's the bitch.

Consider that there are dozens of even kinda-popular Lisps, despite them not having much syntax in the first place. They don't just differ in the function names.

You seem to be imagining a ‘universal-ish intermediate language’, so to say, that would permit people with different experience to quickly adapt to an environment of a different language. Not possible, because a) semantics is where the difference is, you'd need a full-blown language translator, which still can be lossy or incomplete (e.g. no types in your standard Lisp); b) one would still have to deal with all the non-language parts of the environment, like the library, packages, the build system.

> You seem to be imagining a ‘universal-ish intermediate language’

To clarify, I do not mean that each developer would be using a different language -- they would all be writing "Python", they'd just using a superficially distinct syntax that each finds most pleasant/familiar/ergonomic.

> Remembering whether blocks are delimited with braces, indentation or e.g. ‘then/end’ is a no-brainer.

I tend to find the same, at least after an initial learning phase. In fact, I sometimes find distinct languages with similar syntax confusing since I subconsciously expect the same behavior and packages too.

Having said that I've taught and TA'd Scheme, Java, Jess, and block languages like Scratch & Alice many years ago, and in those cases a significant number of students found unfamiliar syntax to be a major hurdle, especially if it was one of their first two programming languages.

Also, judging by the volume of online complaints about Python's semantically meaningful indentation, Ada's array indexing, and Lisp's parentheses these kinds of conventions do seem like a concern even for experienced developers.

> one would still have to deal with all the non-language parts of the environment, like the library, packages, the build system.

Right, that would be the point, to be able to use a language's library/packages/build system without necessarily using its original syntax.

Well, doing this on top of a single language is more feasible, however the Python-Lisp pair is still not the best choice due to Python's strict semantics of separation between statements and expressions: when you generate an AST from Lisp, you'd have some sorta verbose shims in place of Lisp's expression-ic statements—precisely ones that Hy now creates. And vice versa, a coder's normal Python might not map cleanly to Hy.

If you're free to invent the languages in the first place, it would be kinda trivial to make several with the same semantics but different superficial syntax—basically just substitute one bunch of characters for another, no need to go via AST even. And these languages can then be transpiled to a single language of your choice, be that Python or JVM bytecode—if the semantics are made to be close enough.

What about a Lissp-Hebigo pair? https://github.com/gilch/hissp#hebigo

Hissp takes a different approach than Hy. Where Hy has to use shims to pretend statements are expressions, Hissp just targets the expression subset in the first place. (Actually a somewhat smaller subset than that if you're not injecting any raw Python: literals, lambdas, identifiers, and calls.)

There are code which takes the Python AST and attempts to produce the correct Python code. It has a few issues if i recall correctly but it should mostly^tm work.

https://github.com/berkerpeksag/astor

That capability is in the standard library now: https://docs.python.org/3/library/ast.html?highlight=unparse...

Basically what you’re suggesting is a kind of AST decompiler (in an IDE front-end?) that allows developers to view and edit the code in an essentially arbitrary syntax?

Yes. They'd be using the same language, but see different syntax styles.

If you do not use the contributed “let” macro, then auto generated Python code from Hy source code looks fine. If you look at the GitHub repo for the Hy book I wrote, you will see a Makefile target for generating Python code from the Hy examples: https://github.com/mark-watson/hy-lisp-python

EDIT: not a Makefile target, I used a shell script create_python_source_from_hy.sh

See hy2py: https://docs.hylang.org/en/master/language/cli.html#hy2py

See JetBrains MPS: https://www.jetbrains.com/mps/

The "Projectional Editor" might be the kind of thing you're talking about. The meta language is stored as syntax trees rather than text and can be rendered in different ways.

I worked through the first four chapters of The Little Schemer in Hy a bunch of years ago: https://github.com/andybp85/hyLittleSchemer

I moved on to Racket shortly after (which I sadly don't use nearly as much as I should these days), but that work definitely made me a far better programmer!

The interesting past threads appear to be:

Hy – a Python based Lisp dialiect - https://news.ycombinator.com/item?id=24987764 - Nov 2020 (1 comment)

A Lisp Programmer Living in Python-Land: The Hy Programming Language - https://news.ycombinator.com/item?id=22361821 - Feb 2020 (3 comments)

A week with Hy - https://news.ycombinator.com/item?id=20645776 - Aug 2019 (27 comments)

Hy - https://news.ycombinator.com/item?id=20605660 - Aug 2019 (141 comments)

Some thoughts on hylang (2017) - https://news.ycombinator.com/item?id=19446381 - March 2019 (24 comments)

Hy: A Dialect of Clojure Embedded in Python - https://news.ycombinator.com/item?id=19277272 - March 2019 (13 comments)

Hy – A Lisp-flavored Python - https://news.ycombinator.com/item?id=14909786 - Aug 2017 (215 comments)

Hy - https://news.ycombinator.com/item?id=12893703 - Nov 2016 (5 comments)

Show HN: HyREPL, Hylang nrepl server - https://news.ycombinator.com/item?id=9869478 - July 2015 (7 comments)

Live-Coding Blender with Hy - https://news.ycombinator.com/item?id=9850058 - July 2015 (30 comments)

Hy – A dialect of Lisp that’s embedded in Python - https://news.ycombinator.com/item?id=8696975 - Dec 2014 (77 comments)

How Hy backported “yield from” to Python 2 - https://news.ycombinator.com/item?id=8641126 - Nov 2014 (22 comments)

Hy, a Lisp that compiles to Python - https://news.ycombinator.com/item?id=7214400 - Feb 2014 (60 comments)

Hy: The Logical choice - https://news.ycombinator.com/item?id=7123395 - Jan 2014 (12 comments)

The State of Hy - https://news.ycombinator.com/item?id=7069781 - Jan 2014 (16 comments)

Try Hy - https://news.ycombinator.com/item?id=6700103 - Nov 2013 (64 comments)

Hy - dialect of Lisp that's embedded in Python - https://news.ycombinator.com/item?id=6245191 - Aug 2013 (1 comment)

First steps with Hy, the Pythonic Lisp - https://news.ycombinator.com/item?id=5924709 - June 2013 (28 comments)

Others?

Pure Data (the venerable audio software, sibling of Max/MSP) has its package manager (for plugins) written in Hy. I found that to be a bizarre sighting of it in the wild.

Dear JetBrains, can we have Hy supported in PyCharm please?

Somebody should try the Mulisp approach. There was simple set of rules to transform Algol-like syntax to Lisp syntax.

You would not need to learn Lisp of any kind, you just add some parenthesis to a Python line and it becomes linked list data for a macro definition. And with one-to-one mapping, we can transform the macro output to back to regular Python line.

Not sure how close Hebigo gets to that idea: https://github.com/gilch/hissp#hebigo

Lisp -> symbolic AI

Python -> statistical AI

Lisp + Python -> ???

Looks neat :)

One thing I remember from trying it about 2 years ago, was that its syntax conventions etc was similar to clojure but different in some aspects, made it a bit of a chore for someone used to Clojure to keep track of the differences while using Hy.

Sorry about that. I was an early contributor and a Common Lisp programmer in my spare time when I worked on the early iterations of Hy.

I think it's been mostly papered over now but you still have the :keyword named parameter.

Somebody took the Make A Lisp challenge (https://github.com/kanaka/mal) quite serious :D

There even a mal in Hy! https://github.com/kanaka/mal/tree/master/impls/hy

I played with Hy a few years ago and loved it. You can even use Numpy!

I never went much further than surface level stuff, but I wrote about it and would love to jump back in one day.

This is so cool! I love it. I would only have to convince my boss to allow me to use it in some modules of our python project....

Did they finally get `let` working? I remember it wasn't for a long time, which seems like a deal-breaker for a lisp...

Yes, but you have to (require) it: https://docs.hylang.org/en/master/api.html#module-hy.contrib...

Can anyone tell a story from using Hy in production?

Lisp-1 is scheme.

But guess basically a Python ast generator ... ?

Yes and yes.

This isn't an "actual" Lisp, it's just Python with S-expressions.

What else does it need to be a Lisp?

Reply on news.ycombinator.com