Why Lisp?

  https://github.com/slburson/misc-extensions

I admit it wouldn't work that well for 'n-at-a-time', but here's a closely related example:

  (defun alist-to-plist (alist)
    (gmap :plist nil (:alist alist)))

This is the map-reduce model of iteration -- obviously not with support for distributed computation, but nonetheless useful for programming in the small. And far more elegant, to my eyes, than that LOOP monstrosity :-)

Soviet-era authority figure: "Western-style freedom has its good points---if applied judiciously".

If you're a proper Lisper, you use macros like it's going out of style, and other proper Lispers love your code for it.

All programs have their own dictionary of whatever it is they define, whether it be macros or variables. You can no more understand a function call just by looking at it than a macro call. (Should functional decomposition be introduced "judiciously" into large, monolithic blocks of code that have everything "at a glance" in one place?)

Can be said of anything so it really doesn't say anything at all.

This is could be a good criticism in theory, in practice

you can observe how it works out. Fukamachi uses @ everywhere, attila-lendvai uses his def-star library ej (def (class foo) ...) instead of (defclass foo ...), some people use iterate, etc and all is good.

People can write shit code sans macros no problem. I'd rather have them than don't. After all macros embody the whole point of programming[0]

[0]: http://axisofeval.blogspot.com/2011/05/why-of-macros.html

And? This is true of code in languages without macros, too. You have too learn all the vocabulary, all the types they use, all the functions, all the structure of the program— macros are just a tool for taking these domain-specific things and packing them into a denser syntactic abstraction. A well-written macro improves readability in both the short and long terms.

I mean, say I write a couple of macros to deal with a database and I write code like

    (with-sql-connection ("localhost" ...)
      (select *
         from  "customers"
         where (= "lastname" "Smith")))

    When reading other people's code you now effectively
    have to learn what "language" they use too.

Paul Graham's essay "Programming Bottom-Up" explains it really really well:

"Experienced Lisp programmers divide up their programs differently. As well as top-down design, they follow a principle which could be called bottom-up design-- changing the language to suit the problem.

In Lisp, you don't just write your program down toward the language, you also build the language up toward your program. As you're writing a program you may think 'I wish Lisp had such-and-such an operator.' So you go and write it. Afterward you realize that using the new operator would simplify the design of another part of the program, and so on. Language and program evolve together. Like the border between two warring states, the boundary between language and program is drawn and redrawn, until eventually it comes to rest along the mountains and rivers, the natural frontiers of your problem.

In the end your program will look as if the language had been designed for it. And when language and program fit one another well, you end up with code which is clear, small, and efficient."

http://www.paulgraham.com/progbot.html

    On the other hand, this really hurts readability.

In any language, you can think of programming as building mini languages to solve problems. You have nouns and verbs - types/classes/instances and methods/functions/operators. Lisp gives you those parts of speech and also lets you manipulate the fundamental grammar as well!

Paul Graham addressed this in On Lisp (page 59) [0]:

>So yes, reading a bottom-up program requires one to understand all the new >operators defined by the author. But this will nearly always be less work than >having to understand all the code that would have been required without them. >If people complain that using utilities makes your code hard to read, they >probably don’t realize what the code would look like if you hadn’t used them.

>Bottom-up programming makes what would otherwise be a large program look >like a small, simple one. This can give the impression that the program doesn’t >do much, and should therefore be easy to read. When inexperienced readers look >closer and find that this isn’t so, they react with dismay.

>We find the same phenomenon in other fields: a well-designed machine may >have fewer parts, and yet look more complicated, because it is packed into a >smaller space. Bottom-up programs are conceptually denser. It may take an effort >to read them, but not as much as it would take if they hadn’t been written that way.

[0] http://www.paulgraham.com/onlisp.html

https://github.com/hylang/hy

http://docs.hylang.org/en/latest/language/api.html?highlight...

Note that you can program the CL system by expressing yourself in Python:

https://common-lisp.net/project/clpython/

Common Lisp is not only a language, but also a platform (analogous to Mono or JVMs). It has a model of computation: programs expand fully down to a set of special forms, which then compile. You can build languages on top of this.

    (defmacro unless (test &body body)
      `(if (not ,test) ,@body))

Lisp(s) are certainly better than Python in every way, except when it comes to successful projects completed.

One thing I __really__ miss from working in lisp is the idea that I can reload things in the repl. In Python, once I've imported something, I can't really redefine it without pasting in the definition, which makes iterating on a class definition much harder. In Lisp, I can hit a key in my editor and the running REPL gets the new definition, and I can start working with it (or rewriting tests, etc).

The power of the REPL in lisp is __amazing__. I love me some IPython (it's awesome), but there just isn't the same tight integration between that and a running system. The "default" Python likely has all the tools you need to do that, but it just isn't presented as the Way you Do It.

Django's auto-reload when code changes is an example of this. The trouble is, I can't get to a REPL easily within that, without invoking ipdb. I don't know how to integrate my editor with the Python process in a similar way, etc.

All that said, I still love coding in Python. Hy makes me excited, but then I just started writing python-with-parens and wasn't sure what I had gained. :)

0: I'm sure there is someone who can give a counterexample, but I cannot thnk of any.

Also when you learn some Lisp, you start to realize how often people are greenspuning[0] things in their projects.

[0] - http://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule

* Another programmer's 15 function API, and a document on how to use it properly?

* Or another programmer's 15 function API, along with three macros which use the API properly, and capture all the scenarios I need based on a couple of examples.

Macros allow for the arbitrary evaulation of it's arguments (rather than the standard left to right order before a function call), allowing you to do syntactic extensions without the added boilerplate that functional languages can require.

Essentially, each macro allows you to define a mini-language that is parsed by the compiler that returns code that is then compiled. It takes a while to groc, but once you do you can never really go back.

The advantage of CL macros is you can use functions written for your program to use at run-time during compilation-time/macro evaluation too.

A LISP macro is a syntax transformation. It lets you write code in the way you want to, instead of whatever level of abstraction you used to have.

I'm not sure about 'partially formed' code output, but 'partially formed' input is definitely possible. The way to invoke a LISP macro need not be valid LISP. In short, LISP macros excel at creating domain-specific languages.

Say Clojure forgot to ship with the boolean "or". "or" should evaluate its arguments one at a time (to allow for short circuiting) and return the first non-false value. You could do this with functions, but you have the source-level overhead of manually wrapping everything, and performance overhead of defining and passing an anonymous function for each argument. With a macro, at compile time you just translate the "or" macro into a simpler form that uses "if". This is how Clojure actually implements "or":

    (defmacro or
      "Evaluates exprs one at a time, from left to right. If a form
      returns a logical true value, or returns that value and doesn't
      evaluate any of the other expressions, otherwise it returns the
      value of the last expression. (or) returns nil."
      {:added "1.0"}
      ([] nil)
      ([x] x)
      ([x & next]
          `(let [or# ~x]
             (if or# or# (or ~@next)))))

  with-open-file(filename, lambda file: do stuff with file)

  with-open-file (filename):
     do stuff with file

When you write with-open-file as a macro, it could be implemented as a higher order function, or it could be implemented as a low level set of GOTO statements. It doesn't matter. The macro abstracts away the low level detail, just providing the most natural way for you to use the construct. It might not seem like the macro is doing much in that particular example, but a construct that defines a class (like defclass) is something you can write as a macro, which can expand into functions that do the actual defining. You could write a class defining construct as a higher order function, but then you'd have to constantly worry about how your construct was implemented as a function. instead of just writing something natural like

  defclass tiger (animal):
    age init-value: 0
    name type: string

  defclass(name='tiger', inherits-from = find-class('animal')
           slots=['age, name'] ...)

Yet, with macros I have this:

  ./txr -p '(mlet ((x (lcons 1 x))) x)'
  (1 1 1 1 1 1 1 1 1 ... nonterminating sequence of 1's ...

The special mlet ("magic let" or "mutual let") construct has allowed the expression which initializes s, (lcons 1 x), to refer to x.

This works because both mlet and lcons are macros. The lcons macro (rather, the code generated by the macro!) returns a lazy cons cell, without immediately evaluating its arguments 1 and x. In the case of x, this is a damn good thing because x is not yet initialized! When the lazy cons is accessed (when the list object is printed), the evaluation of x takes place. By that time, x holds the lazy cons cell, and since the variable x is in scope of the argument x in (lcons 1 x), the lazy cons is able to force, setting its CDR field back to itself, creating not a lazy list, but a circular list.

With lcons, I we can make a fibonacci function quite similarly to how you might do it in Haskell:

   (defun fib2 (a b)
     (lcons a (fib2 b (+ a b))))

Here, we have a marriage between a lazy data structure and a macro. Without lazy conses, the lcons macro would have no target language to expand into. Without the lcons macro, lazy conses can't be used in the above convenient way; we would have to write fib2 in terms of the macro expansion:

  $ ./txr -p "(sys:expand '(defun fib2 (a b)
                             (lcons a (fib2 b (+ a b)))))"

The circular list mlet, when fully expanded looks like this, by the way:

  $ ./txr -p "(sys:expand '(mlet ((x (lcons 1 x))) x))"
  (let (#:g0001) (sys:setq #:g0001 (cons 'sys:promise
  (cons (lambda () (make-lazy-cons (lambda (#:lcons-0046)
  (rplaca #:lcons-0046 1)
  (rplacd #:lcons-0046 (force #:g0001))))) '(delay (lcons 1 x))))) (force #:g0001))

    if (log.isDebugEnabled()) {
        log.debug("expensive" + debug + message);
    }

However, in a language w/ macros, you just say:

    (debug (str "expensive" debug message))

https://github.com/clojure/tools.logging/blob/master/src/mai...

Each answer exposes one or more facets of macros. Indeed, there is not just 'one' thing that make them worthwhile.

Consider a classic pattern of a with macro:

    (with-mutex-held-macro (some-mutex) (do-stuff))

    (call-with-mutex-held-hof some-mutex (lambda () (do-stuff))

I think that e.g. generalized references (i.e. the common lisp setf macro) are not possible with HOF, but HOF is more common in pure languages where assignment is eschewed anyway.

Mark Jason Dominus (author of Higher Order Perl) had some good comments on lisp macros as well:

http://lists.warhead.org.uk/pipermail/iwe/2005-July/000130.h...

But since macros operate at the syntax level, they can do things functions can't do. For example, they can generate and manipulate declarations. Say you're working with abstract syntax trees (assembly trees in a CAD app might be another example). These trees are built from nodes, where each node is of some class corresponding to a syntactic construct: if-statement, addition-expression, etc. etc. There's some functionality you want to have on every node class; a common example is a "children" method that gathers up all the node's child slots into a set and returns it. It is very convenient to have a 'define-node-class' macro that automatically generates the 'children' method, so that when you add a child slot, the method is updated automatically; there's no need for manual effort to keep them in sync.

In this case, the macro is expanding to multiple top-level declarations: the class declaration along with the method declaration (probably, in practice, several methods). Higher-order functions don't begin to let you do stuff like this.

    function SpecDeclP()Parser{
      return Bind(SeqRight(MyKeywordP, IdentifierP), function(nm interface{})Parser{
        return Bind(BetweenParensP(IdentifierP), function(parm interface{})Parser{
          return Bind(SetSwitchUserStateP, function(_ interface{})Parser{
            return Bind(BlockP, function(bod interface{})Parser{
              return Result("func " + nm.(string) + "(" + parm.(string) + ")" + bod.(string))
            })
          })
        })
      })
    }

    doparse{
      nm   <- SeqRight(MyKeywordP, IdentifierP)
      parm <- BetweenParensP(IdentifierP)
      SetSwitchUserStateP
      bod  <- BlockP
      Result("func " + nm.(string) + "(" + parm.(string) + ")" + bod.(string))
    }

Macros can fundamentally change code. Codewalkers. Back when OOP was a big thing (probably still is), Lisp programmers would probably amuse themselves by making object-oriented extensions to the language via macros, and sending them to each other.

This means that you are empowered, not just a language implementer. (First-class functions are important, and typically a better idea than "Guess I'll write my own macro!" but they only go so far.)

Lisp supports first-class, high order functions. In fact, it was probably the first language to do so.

Macros are different. And they're not just "expanded." Think of macros as full blown Lisp functions that run at compile time and generate new Lisp code, that is itself compiled at compile time. Since Lisp code is represented as Lisp's list data structure, it's super easy. The macro system does allow expansion/replacement (like C's preprocessor), but that's just scratching the surface.

This can provide an important speed improvement for complex macros or code used often, in tight loops or frequently called functions (it's like using inline methods in .h files in C++).

In case of using high-order procedure all its arguments will be evaluated in order before application of the procedure while in case if a macro you could explicitly define all the transformations (evaluation rules) for each argument. This is why it is called a special form - it has its own evalustion rules.

Shortcircuiting if or and are canonical examples.

So, with macros you are extending Lisp with new special forms.

No. How would that be the case?

> And making maintenance more difficult

If used poorly, sure; the same is true of most features of most languages.

> (aren't you re-inventing a new language with macros?)

You are extending the existing language with them, much as you are when using a language with user-definable functions rather than only builtins.

No. How would macros break homoiconicity? They expand into atoms and lists (and other datatypes), the same stuff of which macro-free programs are made.

>And making maintenance more difficult?

No. Unless you intentionally write unmaintainable macros. It's the same as if you write unmaintainable functions, classes, etc. They're just abstracting a different thing— syntax.

>aren't you re-inventing a new language with macros?

No, you're expanding the language; you're just expanding the grammar instead of the vocabulary.

>Where does the "First rule of the macro club" coming from? When should you break it?

What is the "first rule of the macro club?"

If used right, macros make things much more maintainable. If you have a hundred nearly-identical codeblocks, where only (say) a string constant is varying, and suddenly you need to change what those blocks do, bam, you've got a hundred blocks to change. If those blocks had been refactored with a macro, all you have to do is change the macro once. It also prevents the new guy from coming and doing a bandaid fix to line #72 making it slightly different from the 99 lines around it and making your life a living hell.

I haven't bought into this "Lisp macros are the best thing since sliced bread" idea, but without a doubt they are infinitely more awesome than C macros.

    Essentially programmatically writing the program -
    is that what Lisp's macros are?

Think of it this way. Programming languages have built-in control flow operators like if-then, do, while, and for. Macros let you write your own operators - that work just like the ones built into the language itself.

Here's an example. Let's say you want to print a list of names.

In a language without an operator specifically for going over lists - and without macros - you may have to write:

    ListIterator iterator = namesList.GetListIterator();
    while(iterator.HasMoreItems()) {
        string name = iterator.GetNextItem();
        print(name);
    }

    forEachItemInList(string currentName in namesList)
    {
        print(currentName);
    }

And there's also the subjective part. Having to write code like in the first example just isn't satisfying. You have to repeat yourself constantly. You may think, "Look, I loop over lists all the time. And it's always the same pattern: I call GetListIterator(), I do a 'while' loop while it HasMoreItems(), then I call GetNextItem(), etc. Why can't I just inform the compiler of the general pattern - and then tell it how to fill in the blanks when I need to actually loop over a list?"

And that's what macros let you do.

The fundamental unit of Lisp is expressions of the form (FOO ...). There are three basic cases.

1. FOO is one of a finite list of special forms. Those are the irreducible base of that language.

2. FOO is a function. In which case the ... is evaluated as Lisp (possibly including doing things like expand out other functions) before being passed to that function.

3. FOO is a macro. In which case the ... bit is turned into a list of things, but NOT evaluated, passed to the macro to be rewritten, and then the result is turned into an expression that is then evaluated by the same rule.

This is not completely true. There is a preprocessing step called reader macros that can do things like turn `(this expression) into (QUOTE this expression) to save typing. But those are very sparingly used.

    Macros are what made lisp, not parentheses.

About Lisp, people always say "Macros are great" and "Code is data" and "Data is code", but it's hard to see what they mean without good examples. I mean, you can write code that writes code in any language that has a `print` statement. And obviously code is data, so what is that aside from some pseudo-philosophic BS?

There's a lot of discussion in this thread about macros - so here's an example of Code and Data Being One, for those that are unconvinced. I hope it'll shed some light.

I have a slang dictionary website. It's backed by a database now, but it used to be statically-generated HTML. I represented the data as XML. It looked something like this:

    <Term term="slick">
      <PartOfSpeech pos="adj">
        <Definition>
          impressive.
        </Definition>
        <Definition>
          smart.
        </Definition>
      </PartOfSpeech>
    </Term>

XML is a pretty verbose format, so - this being a Lisp example - we could probably save some typing if we represented the data as an S-expression. The above XML would become something like this:

    (Term "slick"
      (PartOfSpeech "adj"
        (Definition
          "impressive."
        )
        (Definition
          "smart."
        )
      )
    )

So, let's lowercase the node types, use hyphens rather than camelCase, and remove the unnecessary line breaks. Now the data looks like this:

    (term "slick"
      (part-of-speech "adj"
        (definition "impressive.")
        (definition "smart.")))

Now we need to write the Lisp code to convert that S-expression data into the appropriate HTML output.

We'll need some Lisp functions to handle the nodes and their attributes (such as the definition text and part of speech) and write the HTML for them. We could use an S-expression parser library to load the data, and then walk through it and call those functions. But that's not necessarily the best way. We can simplify it by creating exactly 3 functions that take some arguments and output HTML: term, part-of-speech, and definition.

Since Lisp code is - like the data - also represented as S-expressions, once we've written those 3 functions, the data is literally executable Lisp code.

    only if you don't look too closely

    what about break and continue?  They are usually
    supported by native foreach construct but are
    difficult to build with macros..

But if your looping macro uses a native looping construct or another macro that supports break and continue, then your looping macro will inherit that support, provided that you're careful when writing the macro to do nothing that will break it.

If your looping macro doesn't use a construct that supports break and continue, then you're still in luck. Continue I described above. Like continue, break is also GOTO by another name. You've got TAGBODY, GO, CALL/CC, etc.

Why do you think that supporting break and continue in macros is difficult?

If you do like vim, slimv is a really great option for lisping.

Basically I have a horizontally split 'screen' session, code in the top and repl in the bottom, and I just ctrl+[c+c] to send paragraphs from the top to the bottom. I don't remember if vim-slime comes with it or if I augmented it, but I also do ctrl+[c+f] to send the current Lisp/Clojure form, ctrl+[c+l] to send the current line... Or just ctrl+a+tab to switch screen windows and type in the REPL directly.

Usually example where I see people rewrite XML to s expressions (like in this thread) are very lossy -- its easy to be pretty by throwing away most of the information!

> Those two things are not mutually exclusive.

I beg to differ. I just replied to someone else about this: https://news.ycombinator.com/item?id=9509110

I agree with your last paragraph, though. There is a timelessness about S-expressions.

As a side point, I would add that the distinction between strings and symbols is important, and neither XML nor JSON has it.

http://eli.thegreenplace.net/2012/03/04/some-thoughts-on-jso...

I do it too in python via ipython. Maybe what the author meant is "Why exploratory programming using a REPL is great".

I run Linux on my web servers but develop from a Windows desktop. CCL works great on both.

I don't think the developers have worked around that, though I have seen some conversation about it in the past. My limited Google searches on the subject today didn't suggest that anything has changed since the last time I looked at it.

Edit:

Oh, duh, I already said all of this about 4 1/2 years ago:

https://news.ycombinator.com/item?id=1804415

Visiting the system requirements page again suggests the limitation still exists:

http://trac.clozure.com/ccl/wiki/SystemRequirements

This line doesn't hold on its own but as the ending of the article it was very powerful.

I had heard that extra power can make it easier to do things wrong, but the idea that extra power can cause problems even when you do things right is fascinating.