Lisp in Your Language | Hacker News Reader

Lisp in Your Language

danthedev.com

215 pointscodecurve76 comments

Oh noes! Don't give people these bad ideas! :)

In all seriousness, I worked with a bright guy who was a lot of fun to work with, but every once in a while his exuberance for this sort of thing would find itself in code that other people had to read/maintain. One particular example, he implemented a small subset of Lisp encoded into PHP arrays and it was used throughout some test code.

He effectively proved you could write PHP in a Lisp style and that PHP programmers overwhelmingly hate PHP flavored Lisp.

Drupal "theme arrays" are not entirely different. They wanted expressions covering what to be themed and how to theme those things and to allow nesting of this information - S-expr flavored PHP. I wish I knew if this had been done on purpose or if it was (just) a sad accident.

I'll guess theme() was inspired by s-expressions, whether consciously or not. The theme() function dates from when Dries was a CS student writing a lot of the code himself.

(It's also worth noting that theme() hasn't worked this way since 2011; it now takes a more PHP-style associative array of named options.)

I'm not sure you understand. theme() appears to just be a pretty simple observable ("hookable") delegator... I mean the extreme flexibility (and great potential for large amounts of slow synchronous processing from abuse) of using nested trees of elements with '#theme' function set as passed to drupal_render().

Oh god, haha. I've been tempted to do this before. I ended up just writing a parser in PHP. The idea was to compile the code to phar and then bootstrap the compiler in Liphp (I chose Liphp because of the mashup AND to tease my mrs about her childhood lisp in an original way).

Before the bracket-syntax for arrays, this would have been awful to maintain.

Greenspun's Tenth Law [0] is a humorous aphorism that applies here.

>> Any sufficiently complicated C or Fortran program contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.

[0] http://c2.com/cgi/wiki?GreenspunsTenthRuleOfProgramming

What would be some recent examples of this. (In the spirit of generosity I will accept a substitution of c++ for c and a reasonably-fortranish language for fortran).

glib offers much of what Common Lisp offers as part of the language (and more, of course).

I'm not horribly familiar with glib, could you expand?

glib[1] is a library for C which provides things like portable file I/O (in Common Lisp); error reporting (in CL); hook functions (part of CL); a reader (in CL); a lists, sequences, strings & hash tables (in CL) and so forth. Common Lisp mostly has a fuller implementation of all of these.

It also has some things CL doesn't provide as part of the language: threading primitives; URL-handling; advanced calendrical calculations; basic crypto and so forth.

[1] https://developer.gnome.org/glib/

Hm, I don't think you can say that an optional, third party library is a "part of a language". Each language has a some standard library, which can be considered "part of a language", but other libraries shouldn't be. This way you could say that all npm, pypi and gem packages are part of the relevant languages.

Other than this, however, it's a good example.

Scala the language ;). Or every time someone reinvents "programming in XML" (and yes, so-called "behavioral trees" in game development, at least in their basic form, also count).

How is scala a large c-ish or fortran-ish project?

Greenspun's Tenth Rule is a joke (there aren't even any other rules numbered 1-9). It's not meant to be taken literally.

The point is that low-level languages like C and Fortran (and today, Java) are underpowered, and any sufficiently large project will evolve to the point where someone builds a (often vaguely Lisp-like) scripting language on top of it to allow developers to raise the level of abstraction, maybe without even realizing that's what they're doing.

I have seen this in action, a lot of companies build internal, proprietary scripting languages on top of their low-level programs. They may start out as configuration files, a build system, preprocessor macros, or a DSL, but over time as features get added they accidentally become Turing-complete, and eventually someone writes an interpreter and now you've invented something that serves the same purpose as Lisp, only worse.

More here: https://sites.google.com/site/steveyegge2/the-emacs-problem

> will evolve to the point where someone builds a (often vaguely Lisp-like) scripting language

No, that's not what it is about. It's not about programs needing a scripting language.

It's about the program (its framework, libraries, architecture, ...) itself. It will acquire features like:

  * late binding
  * symbols
  * dynamic data structures
  * serialization format
  * a plugin architecture with loading of code
  * advanced error handling
  * first class function objects
  * automatic memory management
  * specialized numeric datatypes (large integers, ...)
  * a dynamic object system
  * more...

The implementation language will be extended by some framework, software layers, libraries with such features ... so that over time it will implement many of the standard features of Common Lisp

Note that platforms like the JVM, .net and others nowadays provide many of that (and much more). Also language on top of those do: Java/Scala/... See for example how for Java dynamic loading of code has been implemented on top of custom class loaders.

The yolo PHP framework deliberately invokes this rule:

http://yolophp.computer/

I discovered recently that one part of the Pusher javascript library does basically this (using arrays of javascript strings, just as in the article). And this isn't test code, it's the actual code they use for connection fallback logic. Utterly bizarre.

https://github.com/pusher/pusher-js/blob/2.2/src/defaults.js...

parser in https://github.com/pusher/pusher-js/blob/2.2/src/strategies/...

I wrote that code, there were some reasons why we did it that way. First, before the 2.0 release, we ran several experiments with the connection strategy to figure out the fastest way for the clients to connect to Pusher. We haven't had much experience with HTTP fallbacks, so it was important for us to make the learning process easier. This DSL helped a little bit.

The main reason was that we were thinking about helping clients with server-supplied connection strategies. We could gather some metrics and tell the client in advance that, for example, WebSockets won't work due to some network issues. We also thought it could be useful for orchestrating migrations and working around potential load balancing issues. Injecting JavaScript was out of question, so I wrote this JSON DSL for connection strategies.

We haven't done that and it's probably never going to happen. The Internet has moved on, we dropped support for some ancient browsers, connectivity has improved and we haven't really felt a need for that feature, to be honest.

I agree that at the moment it's unnecessary.

Thanks for the explanation. Interesting to hear the reasoning behind it.

That sounds really horrible, but as a lisp enthusiast that worked on a substantial live system in PHP for a couple for years, I do believe it's possible to write a fair amount of PHP in a functional style. Making use of array reduce, map and filter for example.

It is possible. I actually ended up doing it, partially because I was sick of PHP already and partially because I was the only programmer on the project anyway. array_map and family, however, don't make PHP a scary Lisp. Try writing something along the lines of destructuring-bind "macro", on arrays; I'm still both happy about how it worked and happy that noone else will ever see it; non-lispers would probably gouge their eyes out.

Why did you stick with PHP, if you were the only programmer and clearly prefered other languages? Too much legacy code?

Because the job came easy and I liked it initially. With money inflow temporarily secured I focused on dealing with other areas of my life. I didn't notice how much I'm burning out at first. By the time I did, I started having financial difficulties, because my salary started following my work performance, which was decreasing because of the progressing burnout, which... well, I managed to escape that feedback loop after spending over an year in it, but not unscratched. So basically at first I didn't leave because I liked the stability, and later I couldn't because of a sense of duty.

I was originally in a team of 4-5 programmers, but the company slowly got rid of everyone except me. At that point I tried to leverage the situation and institute some changes in the programming languages, web stack and workflow automation; most of them were rejected by the boss on the grounds like "sure, we could switch to something better - like say, Ruby - but Ruby programmers are much more expensive than PHP."

It took me a lot of work and pain to eventually leave. My two biggest mistakes:

1) Not noticing when was the best time to go. There was a moment when the old project ended and they wanted to get me into a next one. Building it solo. From scrapped bis of metal. I should have left right there.

2) Being too afraid of changing jobs. I know that with my skills I can land a new programming job very quickly, but the process of job interviews scares me.

Oh, and 3) Singing that exit-level contract that ended up with me working my ass off in the evenings to finalize some contracted components.

Thanks for the write-up!

Job interviews are a nuisance, but easily trainable. :)

Related: Scriptor has created what some would call an abomination, but I consider a hell of a lot of fun to play with -- Pharen, an entire Lisp in PHP :)

http://scriptor.github.io/pharen/

Another example is the somewhat obscure language LPC [1], which has a C-like syntax except that you write anonymous functions (called "lambda-closures") in an ugly Lisp-ish DSL [2]:

      x=filter(a,lambda(({ 'o }),
                              ({ (#'>),
                                 ({ (#'call_other),
                                    ({ (#'environment),'o }),
                                    "query_level"
                                 }),
                                 WL_APPRENTICE
                              })));

[1]: https://en.wikipedia.org/wiki/LPC_(programming_language)

[2]: http://lpc.psyc.eu/doc/LPC/closure_guide

Author here, would be interested to hear people's take on the addition of illustrations (illustration is being generous, I know). Do you find them distracting or do they help break up the text?

I think they are nice. I think you made the basic ideas behind Lisp accessible and fun. Good job!

I agree! Lucid approach. The illustrations work for me. I like the approach of using eval immediately, without ceremony.

(Disclaimer: I’m used to Lisp so I’m not really the audience whose opinion matters. But I’m sure I would’ve benefited from reading this earlier. Skimmed it, and am now halfway through a more careful read.)

I like the illustrations, but I love Land of Lisp which is filled with them so... I think I'm already predisposed toward illustrations in text.

I didn't even realize there were illustrations until I read this comment. Had to go back and check.

Haha, that's a pretty clear case for them not being distracting!

Or maybe they were using a reader program which didn't display images, or they were reading HN on Lynx :)

Cool post! I'm curious - where did you get the turtles illustration idea? I've seen it in a funny, obscure book and am wondering if you've read it too.

It's a reference to this : Turtles all the way down

https://en.wikipedia.org/wiki/Turtles_all_the_way_down

Hi sounds like you'd like some constructive criticism so here's MHO.

Images can be great aids in understanding text and making abstract concepts concrete. Also, in many cases the usual metaphor/simile rules apply. For instance, in the phrase he's "fast as a bolt of lightning" we can imagine the speed of the runner and it helps us envision how he "flashes" by opponents. In this case both concepts are known i.e. lighting and fast, but new descriptive meaning is created through the use of simile. Similarly, metaphor takes two known concepts and combines them to reveal a new concept. An example of metaphor, would be "he's an emotional rollercoaster", we know what emotional is and how a rollercoaster behaves, this gives us new insight into the persons high and low spiraling personality.

Essentially, I think good images interact just like simile and metaphor with a text, enhancing, extending, and clarifying concepts. If you go back now and look at your images, I think you'll see that they actually reflect rather verbatim what the text contains, only minimally extend or clarify the text, and at times might make it more confusing (how do turtles enhance our understanding of expressions?). So for these reasons, I found them to be distracting and not helpful, as nice as they actually appear graphically :)

Nice post btw :)

The car illustration was both funny, and helped to show your point in a really concise way. So I'd say +1 to illustrations!

IF is not a function. Functions don't evaluate their arguments. Functions actually get evaluated arguments passed. But IF is a special form, built into the language. Special forms are not functions.

There is something special about special forms. A basic set of special forms is needed and can't be implemented as macros.

> But IF is a special form, built into the language.

It doesn't have to be built into the language; it could be a macro based on COND or even on WHEN/UNLESS. But…

> A basic set of special forms is needed and can't be implemented as macros.

One has to have some special forms in order to build everything else as macros. And in practice, of course, there can be some big efficiency gains to having lots of special forms vice having only a few and lots of macrology.

> It doesn't have to be built into the language; it could be a macro based on COND or even on WHEN/UNLESS. But…

In Common Lisp IF is built in and COND is a macro. One of the conditionals needs to be built-in.

http://www.lispworks.com/documentation/HyperSpec/Body/s_if.h...

> One has to have some special forms in order to build everything else as macros.

For conditionals, one conditional needs to be there, into which the other macro-based conditionals can be expanded.

> And in practice, of course, there can be some big efficiency gains to having lots of special forms vice having only a few and lots of macrology.

Not really. Common Lisp has a minimum of special forms and usually has some of the faster Lisp implementations.

> One of the conditionals needs to be built-in.

That's actually not true. IF can be implemented in terms of LAMBDA.

    (defun _true (then else) then)
    
    (defun _false (then else) else)
    
    (defmacro _if (condition then else)
      `(funcall (funcall (function ,condition) (lambda () ,then) (lambda () ,else))))
    
    ? (_if _true (print 123) (print 456))
    
    123 
    123
    ? (_if _false (print 123) (print 456))
    
    456 
    456
    ?

There are problems with infinite recursion if you do that, no? http://stackoverflow.com/questions/1171252/whats-the-explana...

Nope. That's the reason for wrapping THEN and ELSE inside lambdas, to delay the evaluation until after one of them is selected. Try it.

[UPDATE]

You have to make some minor tweaks to my original code to make it work. Here's a fully tested version:

    (defun _true (then else) then)
    
    (defun _false (then else) else)
    
    (defmacro _if (condition then else)
      `(funcall (funcall ,condition (lambda () ,then) (lambda () ,else))))
    
    (setf (get t 'altboolean) '_true)
    
    (setf (get nil 'altboolean) '_false)
    
    (defun ->altboolean (condition) (get condition 'altboolean))
    
    (defun _fact (n)
      (_if (->altboolean (<= n 1)) 1 (* n (_fact (1- n)))))

Does "fully tested" means it is supposed to work as a replacement of IF in Common Lisp? Or is it just a toy example in a made-up Lisp/Scheme to beat that dead horse?

>> In Common Lisp IF is built in and COND is a macro. One of the conditionals needs to be built-in.

> That's actually not true.

The context was about Common Lisp, not Haskell nor Scheme (of course other languages do things as they wish, including lazy evaluation). There need to be at least one conditional built-in in CL in order to map generalized booleans to {T, NIL}. This conditional could even be eager, but there has to be a construct which selects the approprate value. Which of the other 24 special forms is suitable?

Fully tested means that it solves the problem of infinite recursion cited in the stackoverflow question.

As implemented, it's not quite a drop-in replacement for IF, but it can be made so by rolling the call to ->ALTBOOLEAN in to the _IF macro.

> The context was about Common Lisp

Yes, I know that. The example code is written in Common Lisp.

> there has to be a construct which selects the approprate value

Yes. LAMBDA can do that.

> Which of the other 24 special forms is suitable?

LAMBDA. It can be used both to select the appropriate value and to delay evaluation. LAMBDA is really all you need. Making IF a special form is just an optimization.

In Common Lisp everything not NIL is true.

It's a pretty elementary exercise to fix ->altboolean so that this invariant is maintained.

>> Which of the other 24 special forms is suitable?

> LAMBDA.

I am nitpicking, but LAMBDA is not a special form, FUNCTION/FLET/LABELS are: http://www.lispworks.com/documentation/HyperSpec/Body/03_aba...

> It's a pretty elementary exercise to fix ->altboolean so that this invariant is maintained.

And how do you map generalized booleans to your functions?

Right now, I can think of two possible ways. You can map NIL to #'_false in a hash-table, and use (gethash nil table #'_true) to provide a default value. Or you can also write a generic function:

    (defgeneric ->altboolean (x) 
        (:method ((x null)) #'_false) 
        (:method (x) #'_true))

There could be other ways, but as far as I know, none of them are portable, because even though it is allowed for all defined CL operators to be implemented as built-ins, you generally cannot expect or assume that it is done so (see gethash in CCL and SBCL). As far as you know, and even though it would be terribly inefficient, they could all be implemented only by combining special forms.

If you discard IF, can you perform the elementary exercise using only special forms? I cannot imagine how, but I honestly would like to be shown a working approach.

> I am nitpicking

You certainly are.

> can you perform the elementary exercise using only special forms?

I don't even need any special forms. All I need to do is use the default argument to GET.

    CL-USER 16 > (defparameter a 0)
    A

    CL-USER 17 > (if (foo) (foo) (foo))
    T

    CL-USER 18 > a
    4

    CL-USER 19 > (defparameter a 0)
    A

    CL-USER 20 > (_if (->altboolean (foo)) (foo) (foo))
    T

    CL-USER 21 > a
    5

Given that foo is defined as:

    (defmacro foo () (incf a) '(progn (incf a) t))

4 vs. 5

Does not look like a true replacement for IF in Common Lisp...

4 and 5 are both correct results. Implementations have a lot of flexibility regarding when macro expanders are called.

Welcome to the real world.

I'm not talking about Lambda Calculus and related.

I'm talking about Lisp implementation. Real world Lisp implementation.

Actually, all operators can be macros. Why? Because macros can use some, ahem, special API to generate machine code objects directly. Thus anything that looks like it ought to be a special form can be registered as an actual macro, called by the macro-expander. Instead of producing a tree structure, it produces some code object. The compilation pass knows what to do with that object and so it goes. (We can think of that object as a special form, effectively, but it isn't really a form at all.)

(Also note that Lisp macros don't have to be all written in Lisp. They are just expander functions. When you're bootstrapping a Lisp, you can write macros in the bootstrapping language. So the chicken-and-egg problem of "how do you write the first macros if there are no special forms" doesn't exist. You write the raw functions in assembly, or C or whatever, and register them as macro-expanders.)

Not necessarily: https://en.wikipedia.org/wiki/Lazy_evaluation#Control_struct...

IF can totally be a function in a lazy language.

This is very similar to miniMAL [0]. They even have a miniMAL interpreter written in miniMAL [1] in the Make a Lisp project [2].

[0] : https://github.com/kanaka/miniMAL [1] : https://github.com/kanaka/mal/tree/master/miniMAL [2] : https://github.com/kanaka/mal

You could also take your proto-eval and spit out ASTs into your favorite language for your given special forms and bootstrap a new interpreter from there.

Hy does this for Python (except that its initial implementation started from a parser). It essentially becomes a syntax for manipulating Python ASTs as plain old data-structures instead of classes with terrible interfaces. It's actually a great way to become intimate with the inner workings of your host language.

It's funny how far you can get by sprinkling a few parentheses around.

If JS developers want a concrete example of Lisp's benefits, you only need to look at React's JSX syntax-- it lacks conditional logic and loops. To see why, you kinda have to see how React components are created in ClojureScript's lisp syntax, it's really elegant: https://github.com/shaunlebron/jumping-from-html-to-clojures...

I think JSX is just terrible, glad to see someone else feels the same. The ClojureScript way is much better, though I think SXML style is best: http://okmij.org/ftp/Scheme/SXML.html

I wrote a very similar tool one time:

https://github.com/smizell/geneva

It is Lisp in JSON. I did it for fun, but thought it would a cool way to send instructions across the wire from the server to the client and back. Imagine validation rules that could be ready in any language with really very little parsing instructions.

We're also working on a data format that allows for this use of arrays.

https://github.com/refractproject/refract-spec/blob/master/r...

We use it very similarly to how you would with Lisp, where the "element" may be a function of sorts.

Creating a Lisp is a great way to learn a language. https://github.com/kanaka/mal/ has thorough tests and step by step instructions to help you along the way, definitely recommend checking it out.

Very good post but ewwwww to the way you have to implement it in Javascript.

Still, seems very accessible and useful to people who don't feel like going through the trouble of learning/building a "real" lisp but will read a blog post.

Well, once you've got this much going all you need to implement is a proper READ, and then life is just about perfect.

Compare https://en.wikibooks.org/wiki/Write_Yourself_a_Scheme_in_48_... for Lisp in Haskell.

One of these days I'm going to stumble into something like this in a project. Probably written in Delphi or VB.NET. Not that I use those languages, but it tends to be that the weird stuff tends to appear there. At least for me.

"Not that I use those languages, but it tends to be that the weird stuff tends to appear there."

I guess programmers of those languages must desperately want to escape to another language.

So a Lisp in VB is like fingernail scratches on an old prison cell wall? I like that analogy!

Dunno, the ones I know like them. But they seem to re invent everything. If the universe ever gets reboot a vb.net programmer will be responsible.

I'm working on a data validation pipeline, and I recently noticed that the rules in my jury-rigged rule engine already look a bit like lisp, and briefly considered embedding a tiny scheme. That consideration lasted about a coffeetime.

My solution is going to be turning these rules into something that looks like Python, and feed each line to eval(). I reckon that will make 90% of the rule evaluation engine, which is my brittler code, to just disappear.

Sometimes the most obvious choice is also the best.

No, none of these rules are input by users. All of them come from our repo. We're safe and good.

I've made an embedded language called Plumb, which works along similar lines: http://chriswarbo.net/essays/plumb

Plumb is certainly not a Lisp though; it's actually a syntax for lambda calculus with de Bruijn indices, which just-so-happens to embed easily in common scripting languages (PHP, JS, Python, etc.).

Lisp articles have been making their way to HN's 1st page the whole week.

Keep them coming.

Maybe we can designate this week as Lisp week every year where we revive our collective love for the language. Sure beats shark week in value.

> Maybe we can designate this week as Lisp week every year

For some of us, every week is Lisp week grin

And others only dream that maybe their grandchildren will live such happy times as their grandfathers did. But it's time to go to bed, get some sleep before another day of reimplementing half of CL in Java...

https://xkcd.com/297/

Reply on news.ycombinator.com