Shen: A Sufficiently Advanced Lisp [video] (2014) | Hacker News Reader

Shen: A Sufficiently Advanced Lisp [video] (2014)

youtube.com

112 pointssea6ear45 comments

Also interesting and worthy of note is that this project transitioned to BSD recently, and they are doing it as part of a donation drive.

http://www.shenlanguage.org/shenbsd.htm

I am an amateur Lisper, and like others I whined (perhaps irrationally) that their custom non-OSI only one implementation of the language or else license was scaring people off. Not that the BSD license assuages my concerns about community weirdness (hard to describe, but this ia a very personal opinion and you can dismiss it), but I would like to see if the demand will drive open sourcing the code.

Also very cool is that you can find Shen implementations in more than one flavor: Common Lisp, Clojure, Ruby, and more. Definitely check out the site if you have not yet.

That was a really weird "episode" in the history of Shen, IMO. I mean, the outcome was good and all, but it was really weird to ask for donations to change the license. I don't think there was any real explanation for that other that "we want money"?

I believe it came from a discussion where to draft a new license would require the original developer to pay a lawyer. Someone suggested raising money to pay the developer the money to transition to BSD, a known license, instead of to a lawyer for yet another unknown license.

Wow, that's even weirder than I imagined. Regardless, I'm glad it's been freed due to the generosity of some individuals :)

It's pretty strange to see a "Lisp" with so much syntax. Lots of infix notation, and even semicolons! Usually Lisps have a very simple reader that reads S-expression and a few other things, but here we have this:

  (define map
    { (A --> B) --> (list A) --> (list B) }
    F [] -> []
    F [ X | XS ] -> [(F X) | (map F XS)])

And this is one of the cases without any semicolons.

Shen also uses the atrocious "curried form" of function types, where the function that adds two numbers has type number->number->number, and the reader must do the mental gymnastics to convert it to (number,number)->number (and good luck when it's more complicated than that).

I wonder if you took a language like Haskell and changed some of the syntax to look Lispy if it could masquerade as a Lisp? I'm not saying that's what's happening, it's just hard to wrap my head around a syntax like this that claims to be Lispy.

You don't have to use any of the syntax if you don't want to. You can write directly in KLambda. Here's how `map` is defined in KLambda:

   (defun map (f l)
     (shen.map-h f l ()))

   (defun shen.map-h (f l accum)
     (cond ((= () l) (reverse accum))
           ((cons? l) (shen.map-h f (tl l) (cons (f (hd l)) accum)))
           (true (shen.f_error shen.map-h))))

http://readwrite.com/2010/11/27/haskell-with-lisp-syntax?_es...!

Put this in my "watch later" queue. I'm very interested in a Lisp whose type system is on par with that of Haskell/Ocaml.

EDIT: Especially since Shen is now BSD licensed.

To me, it seems impossible to have both macros and a type system on par with Haskell. Either your macros can no longer do arbitrary textural transformations, or your type system can't reason about macro cases. (Or, I suppose, your type system has to run after the macro transformations, and you could get a type error then just like you could get a syntax error then. But that means your IDE/development environment can't give you any type assistance on macro calls - you have to compile it to find out if it works. (Or, I suppose, your IDE has to run the macro for you and then do the type checking on the resulting post-macro code.))

I am open to being proven wrong, though...

Scala has macros [0], and as you reasoned the type checker has to run after the macro expansion. That's not a huge downside; to the user it still appears to be a single pipeline (parsing -> macro expansion -> type checking -> compilation).

Haskell also has very powerful metaprogramming facilities, like template haskell [1].

[0] http://scalamacros.org/ [1] https://wiki.haskell.org/Template_Haskell

ATS had macros: http://ats-lang.sourceforge.net/htdocs-old/TUTORIAL/contents...

I'm not sure if these made it into ATS2 though.

The second half of the talk (http://www.youtube.com/watch?v=lMcRBdSdO_U&t=17m30s) is about turning a JSON object into a type using macros and meta-programming.

MetaOcaml's type-safe staged metaprogramming is not strictly a macro system, but it is morally equivalent: http://okmij.org/ftp/ML/MetaOCaml.html

> it seems impossible to have both macros and a type system on par with Haskell

You should check out the work being done on typed template Haskell then.

You can have a language which has to be parsed top down, then you can have both complete type info, and arbitrary macros.

Now, I understand that is BSD licensed, but I am bothered by their proud "we support OSI, but never GPL" attitude. I wonder what happened to the original developer, but does anyone know why he has GPL issues? Yes, I know he is not unique in the industry at large, but I rarely see project download pages with GPL with a red line through it.

http://www.shenlanguage.org/download_form.html

The original developers thoughts on GPL have been discussed at length on the Shen mailing list during the BSD licensing donation period. If you look through the archives you'll be able to read their thoughts.

This link may be a start: https://groups.google.com/d/msg/qilang/mVSJIyp-OhM/FjcAOAWUi...

It's unfortunate that the original authors views on licensing takes so much discussion away from Shen itself as it's an interesting language.

So true. I am watching this video and I am very impressed. I am a novice programmer, but from what I read on HN the Shen system hits all cylinders a lot of the more powerful advanced programming paradigms. I am a Shen outsider and there seems to be some much overhead that is not technical when I read about it and it upsets me bc, well, it is a work of art.

I just started studying Java, and found the yet to be certified Java implementation. I am afraid to look at it, lest brains come of my nose.

https://github.com/hraberg/Shen.java

One of the great advantages of Shen is that it is built on top of a very simple Lisp called KL of which you can (at least basically working) implement an interpreter or compiler in any language in a day. And when you did that, Shen will run on it. That's why there are so many targets and that's why most targets are not mature. I find it a great advantage though; you can make a compiler or runtime embedded in whatever you are doing really fast and then optimise it as you go.

Edit: I would say, if you want to practice on implementing a compiler & runtime (or VM with JIT or LLVM frontend etc) KL is a good place to start. It's so easy (again, to get something working; to optimise etc is obviously just as painful as with any other language implementation) that it is a lot of fun and you learn a lot if you never did it before.

Is there any good introductory info on K lambda? I did some quick Googling and Wikipedia searching, I did not find a singular article about this, not general lamdba calculus or something useful to a complete novice like me.

I think this from the Shen site describes it: http://www.shenlanguage.org/learn-shen/shendoc.htm#Kl

Yes, you would use that doc to implement it and then the Shen distribution to test if it works as expected. Unlike Shen, K lambda is quite a simple Lisp implementation, if you know a bit of Lisp it should all look pretty straight forward. It doesn't have any advanced stuff built in; that's all done on top.

Thanks for that link. I might not agree with all views presented, but the question about re-licensing as framed there was new to me. I'm very much pro GPL/FSF -- but when leveraging copyright to achive copyleft, it is important to understand the inner workings of copyright, and as this thread highlights the nitty-gritty of "derived work" and the difference between a liberal license to (re)distribute, versus the right to assert copyright, and thereby re-license.

Once again, types are way overrated. However, Shen is an interesting new Lisp to watch. Needs a more open community, though (like a free book and/or deep documentation).

> Once again, types are way overrated.

Care to expand on this? I'm curious, what is your opinion and preferred platform/languages?

I'm not the parent, but in noncritical applications an accommodating compiler/interpreter will speed up development. Few advocate such an attitude for critical applications, but whether all compile-time checking should be done in terms of types or not is an open question.

Types speed up my development: they obviate a certain kind of unit tests. If you weren't going to write those unit tests without the types, anyway, then it helps because you now catch type mismatches at compile time instead of runtime.

The question is whether types are the most productive area to focus on when developing better compile-time checks. Maybe it would be better to design languages that solve problems by other means.

With Shen you can use them when you want. So you can create things fast and add types / tests / proofs when you want and when it seems prudent.

Have you never observed this discussion before?

> "I'm not the parent, but in noncritical applications an accommodating compiler/interpreter will speed up development."

What does "speed up development" mean? Does it mean "as fast as you can push it to your users"? Does it mean "as soon as it passed QA"? Does it mean "as soon as we receive no further bug reports on it for at least 6 months"?

> types are way overrated.

Please show your scientifically rigiorous evidence. Because it appears very few other people have any (either way), so at the very least you could earn a prize if you shared yours! :)

How about "A Large Scale Study of Programming Languages and Code Quality in Github" by Baishakhi, et al. [1]? That shows a weak correlation between fewer bugs and static typing. There's a stronger correlation between strong typing and fewer bugs.

[1] http://macbeth.cs.ucdavis.edu/lang_study.pdf

I am familiar with that study and while I have serious issues with the methodology, I do not think it says what you think it does[1].

[1] If you think it refutes my point, that is.

> Because it appears very few other people have any (either way)

If there is no evidence either way, yet every other developer raves about types all the time (the way "connaisseurs" rave about fine wine and expensive scotch despite being unable to objectively tell the difference), doesn't it follow that types are overrated?

Wow. Way to bias the discussion by segueing into to wine and/or whisky! We're not discussing neither wine nor whisky, so... maybe you'll dispense with the hyperbole?

What is your actual point and what is your evidence?

That's the point. Here is further discussion of this: http://blog.metaobject.com/2014/06/the-safyness-of-static-ty...

The community is bizarre, I will not lie, but the original developer did release a book quite recently using Shen, although Shen might not be the focus.

http://www.shenlanguage.org/LPC/lpc.html

There are two books you can buy ; the latter one uses Shen; the first one (The Book of Shen) teaches you Shen. If you know Lisp & Prolog, it should not be hard to pick it up from examples and the source code. The book is worth it, I just wish there was a digital version; I think it would sell a lot more.

I've even come to the conclusion that I don't really like dead-tree books any more. I mean, I like them, as a luxury (and I buy used books) -- but since I'll have a reading device/laptop anyway -- any dead tree books are just a waste of resources (trees, print, packaging, transport, storage, transport...).

Now, books have been optimized really, really well -- and actually take up remarkably few resources to make -- but as long as I'll have a device on which to read e-books anyway -- all those resources are wasted.

Just give me a DRM-free epub version, and I'm much more likely to buy your book.

The community is small but open; agree on the docs.

Unfortunately I found shen a pain to get into due to it still being mostly a teaching tool. However, if the community builds some nice tooling around it I will definitely give it another look.

Working on it!

Shen stands above other languages (apart from Idris and Coq) in the area of dependent types.

I haven't looked at the ability to have dependent types in Shen - it'd be great to see some examples. A quick search on the mailing list found:

https://groups.google.com/d/msg/qilang/lokvvf0xLJc/20y8ZkzoN... https://groups.google.com/d/msg/qilang/D3tmhQvfnOE/qcsXhRomF...

and apart from ATS.

Reply on news.ycombinator.com