Every time, I see one of these links or videos, I feel the urge to learn Lisp. But after some time, I lose the motivation. I think that is because I don't know what benefit learning lisp will provide me concretely. Anyone has any suggestion?
That said, lisp is a goldmine / rabbithole crossover. As other said:
- opens for a hackable tool mindset, use lisp on lisp to make it do what you need [1]
- as said in this talk, if you want a dsl, you don't need a parser. Only later if you need others to write it without '(lisp syntax) you can do so. And if all are happy with lisp, you just skip that entirely.
- it's often highly interactive, and value oriented. You get to "touch" the data a bit like material. Nowadays repl's are common, so it doesn't seem special but it was so for 30years.
- lists mindset is different from mutable arrays, saving you immense amount of time in the first phase (by avoiding the vast majority of stupid bugs). When you need speed you can tailor; also lisp implementations are performant (SBCL is in the top 10 of all IIRC)
- functional mindset also open for a lot of new ways to think about problems, leads to other languages like ml/haskell too
- similarly you often get a step in the logic programming world (prolog, kanren) which are very mind opening about "programming computers"
it's not perfect, and if abused you can get stupid code, too cryptic because you used too many weird notation or structure; this is a balance skill you get to learn, lispers are not that asocial, they know when to stay in the human readable most of the time.
REPL's are common, but they're rarely as useful in other languages. Even relatively basic (to Lisp programmers) things are almost universally missing, for example:
[1]> (defun f (x) (1+ (g x)))
F
[2]> (f 4)
*** - EVAL: undefined function G
The following restarts are available:
USE-VALUE :R1 Input a value to be used instead of (FDEFINITION 'G).
RETRY :R2 Retry
STORE-VALUE :R3 Input a new value for (FDEFINITION 'G).
ABORT :R4 Abort main loop
Break 1 [3]> :r3
New (FDEFINITION 'G)> (lambda (x) (* x 2))
;; Now that we gave an fdefinition for G, our (f 4) call can continue, so it
;; resumes execution and completes. Until now, it was just paused--no stack
;; unwinding unless we ask for it.
9
[4]> (g 10)
;; Because we used STORE-VALUE, it went ahead and saved the value we gave in
;; G for future use. If we had used USE-VALUE, it would have finished the (f
;; 4) call using that definition, but G still wouldn't be fbound after.
20
Mutable arrays are ubiquitous in Lisp. Explicitly cdring down lists is mostly only done in introductory textbooks; in practice, most people will use MAP, REDUCE, REMOVE-IF-NOT, etc instead, which all work just as well on arrays or lists, or they might use an imperative construct like DO, LOOP, or ITER (also note that conses are mutable as well; it's not at all unusual to setf a car or cdr or call NCONC). It's not terribly different from what you would do in eg modern Java (aside from the part where none of it requires special support from the implementation and it could all be implemented in user code).
I would also add that learning Lisp will teach you a lot about object-oriented programming. Even things that Java programmers use extensions for and give names like aspect-oriented programming just come built in as part of Lisp's stock object system. But then, Lisp's stock object system is also extremely flexible, especially given the pseudo-standard metaobject protocol. Quoting from Wikipedia about the MOP book:
"In his 1997 talk at OOPSLA, Alan Kay called [The Art of the Metaobject Protocol] "the best book anybody's written in ten years", and contended that it contained "some of the most profound insights, and the most practical insights about OOP", but was dismayed that it was written in a highly Lisp-centric and CLOS-specific fashion, calling it "a hard book for most people to read; if you don't know the Lisp culture, it's very hard to read"."
How?
CL-USER> (remove-if-not #'evenp '(1 2 3 4 5))
(2 4)
CL-USER> (remove-if-not #'evenp #(1 2 3 4 5))
#(2 4)
CL-USER> (loop for x in '(1 2 3 4 5) when (evenp x) do (format t "~&~a~%" x))
2
4
NIL
CL-USER> (loop for x across #(1 2 3 4 5) when (evenp x) do (format t "~&~a~%" x))
2
4
NIL
(defun print-evens (sequence)
(loop for i below (length sequence)
for x = (elt sequence i)
when (evenp x)
do (format t "~&~a~%" x)))
> Even in the 2000s we had to hand write Java Iterators and fear mutation while iterating .. that sort of things.
And even in Lisp, the standard specifies "The consequences are undefined when code executed during an object-traversing operation destructively modifies the object in a way that might affect the ongoing traversal operation."
What languages do you know that have map and filter for lists, but not for arrays? Lisp has them for both, C++ has them for both, Java has them for both; Java didn't used to have lambdas or map, but it had standard lists long before it got either of them. Several decades before Java existed, Lisp had mutable arrays with map and filter. Supporting higher order functions has nothing at all to do with arrays vs lists.
> didn't have lambdas so you end up writing imperative loops
Lisp does have lambdas and imperative loops are still extremely common in Lisp code.
My point was that the functional API is the key difference, not arrays vs lists. Java in 2000 had lists and no functional API, whereas Lisp in 1970 had mutable arrays with a functional API available for use on them. Mapping over a list in Lisp looks like `(map 'list #'f '(...))` (The first argument is the return type; MAPCAR is just a list-specific version of MAP), mapping over a Java-style array looks like `(map 'list #'f #(...))`. Lists vs arrays makes no real difference (if they were in variables instead of literals, you couldn't even tell if it were an array or a list by looking at that call). Java in 2017 similarly has a functional API that can be used on lists or (much more commonly) on mutable arrays.
Now, also pointed out by the sibling post, modifying the underlying driver of your logic is a bad idea. Just like having a printer print on its own circuitry would be a bad idea.
Thus, writing code that generates code, be it at runtime or at compile-time, is downright easy in Lisp. This opens up enormous possibilities not found in other languages.
Also, on regular programming languages, your code executes only at run time. In Lisp, or at least in Common Lisp (and probably on Scheme as well), there are three times: Read time, Compile time, and Run time, and your code can selectively run at one of those three. This also opens up lots of possibilities.
Another feature is that it is really suitable to interactive development in a way that isn't matched by any other language except for Smalltalk. It allows, for example, changing the definition of a function while your code is running.
Now, two features that are particular to some Lisps:
Scheme has continuations. Continuations are like magic teleportation within your code. They are very powerful.
Common Lisp has the Common Lisp Object System (CLOS), possibly the most powerful OOP system out there. It rocks!
Then the only reason you'd have to use macros is if you want to do something at compile time. Like generating specialized code so the compiler can optimize it, instead of reinterpreting the describing data on each invocation. Or like reading the database schema to generate accessors.
That leaves a few other uses: macros as compiler extensions for generating fast code (see "Paradigms in Common LISP" for a lovely example of a parser generator). But heed warnings about premature optimisation.
It leaves macros for novel binding strategies. List comprehensions, do-notation, destructuring-bind, pattern matching, CL's LOOP and macros that anonymously bind a pronoun like "it".
I have no shame in using macros for really succinct control structures where delaying evaluation with a lambda would just look gross (see "or" and "and" macros), but maybe that's just a cry for lazy evaluation :P
The new atoms are supposed to communicate intent faster and more concisely, and decrease the burden of understanding what's going on. That's the role of an abstraction, especially syntactic abstractions.
With that said, I tried to learn about metaprogramming in a number of other languages, and Lisp is the first one with a metaprogramming model that I could actually grok, because the meta-language is equivilant to the language itself. Lisp macros are more or less just plain old code that manipulates data structures.
When I'm working on a lisp project, I give myself a budget of a very small number (often just 1 or 2) of magic macros that I can build for it. Other than that, I only use macros in the core library, or well-known third party libraries that other people are likely to be familiar with.
Macros in other languages are significntly unlike Lisp macros. You should take a look at the Practical Common Lisp book (available for free online) to have an idea on how you can easily leverage Lisp macros for more readable, succint, easy to understand code.
>But if you introduce macros, or code writing code, then you have a much more complex mental model
In Lisp, you use the different tools available (macros, CLOS, readtables, etc) to achieve a simpler translation between the problem domain and your actual source code.
If you are using them to create more convoluted, harder-to-understand code, then "you're doing it wrong", just as you can, for example, write rather clean C code versus code that would win the IOCCC (International Obfuscated C Code Competition).
On Lisp is a comprehensive study of advanced Lisp techniques, with bottom-up programming as the unifying theme. It gives the first complete description of macros and macro applications. The book also covers important subjects related to bottom-up programming, including functional programming, rapid prototyping, interactive development, and embedded languages. The final chapter takes a deeper look at object-oriented programming than previous Lisp books, showing the step-by-step construction of a working model of the Common Lisp Object System (CLOS).
As well as an indispensable reference, On Lisp is a source of software. Its examples form a library of functions and macros that readers will be able to use in their own Lisp programs.
That's not always a good thing -- it can make working with a foreign codebase difficult -- but it's definitely a powerful concept when applied correctly.
But then you hear the arguments in favor of Go maintainability, and it's about copy/pasting being preferable to abstraction and not having too many ways to do things, so everyone's code looks familiar. Also that the extra LOC are more readable to Go maintainers than powerful abstractions. That surprises me, because languages like Lisp, Smalltalk, Ruby and Haskell are all about powerful abstraction capabilities so you can express yourself exactly as you need instead of writing a lot of boilerplate.
Me too.
I don't understand how people in HN vouch for restrictive languages.
What happens as you add more developers to a code base, with larger variance in ability and favored abstractions is going to be important in some cases, and irrelevant in others.
Imagine that you're joining a project. It's been worked on by a team of 100 people for a decade. Half of those people were below-average programmers. Many were newbies in the language, and some were newbies to programming. And you're going to get to try to maintain this code.
Now, do you want it to be written in a restrictive language, or in one that gives developers the ultimate amount of freedom?
I wouldn't join this project, no matter what the language is.
Imagine it's a popular languae. Javascript or C code, for example. There are no true namespacing / packages and modules facilities in JS or C. It would be even worse than the theoretical nightmare you think Lisp would be. (Lisp has extensive namespacing facilities; code can be contained within modules that don't clash.)
If it was Java, you will see wrongly applied Design Patterns, leading to over-complicated, hard to mantain code.
No, thanks. I wouldn't accept no matter the languages.
Newbies to programming should be educated and trained, not incorporated directly into an important project.
Then you will never be employed, because that describes virtually every software project at a for-profit company.
I have had 8 years doing software dev (at a for-profit company) most of them as software development director in command of a 15+ people team, thus, I reaffirm what I said: Newbies should be trained first, and only afterwards included in the projects, and that's what I made sure happened on the team, under my command.
They deserve to be trained first, in an environment where they can make mistakes freely until they feel confident.
- A dumb language and simple code in a big project means lots and lots of code. On such codebase, my biggest issue is keeping track of how things fit together, because there's just so much of it (hint: they don't; people writing it can't keep track of all that stuff either).
- A powerful language and complex code in a big project means dense code. Like in that Lisp codebase, where I dealt with big macro-writing-macros, I would spend an hour with a macrostepper, trying to grok what a single line of code does. But once I did, that line of code (and similar lines in other places) were not a problem anymore, and they compressed what would otherwise be thousands of lines of boilerplate.
Which one I like more? I don't know. Big, old codebases suck, that's a fact of life. But I lean a bit towards "more Lispy" than "more Java-y", because it makes me feel I'm using my brain to actually think, instead of just tedious bookkeeping.
I have seen enough C and Java code to see how creative developers and architects get to workaround the limitations of a restrictive language.
A convoluted code base full of design patterns, indirection layers, DSL and UI tooling for code generation, libraries to simulate language features.
So when a new one comes into the project there is this spaghetti of workarounds in place.
There is a mindset that's popular among programmers which says, "I cannot understand what anything does except by knowing all about its internals." Languages like Go let the programmer keep that mantra instead of understanding the things they use based on a description of external behavior.
There should be only one way to do it. — Python
There's more than one way to do it. — Perl
Do the right thing. — Lisp
But regarding type checking, Lisp (at least Common Lisp) is strongly typed. Really, very strongly typed (for example it will complain about putting a "byte" in a "character" array; or of using an "array" when a "simple-vector" was expected... Lisp is very nitpicky regarding types!), but the type checks happens mostly at runtime. Some checking also happens at compile time, even more if you intentionally include type declarations. (Type declarations are part of the ANSI Common Lisp standard.)
Just a note that this is entirely implementation-dependent. SBCL, for instance, is very good about using type declarations as correctness checks (those that can't be statically verified transparently degrade to runtime assertions) but their exact behaviour isn't specified in the standard; for example, implementations are free to take them as declarations that the programmer knows things the implementation doesn't and to trust them, which could cause weird bugs if they're not correct.
- CL standard defines a pretty decent type hierarchy (not Haskell-level decent, though).
- While not required by the standard, good CL implementations make use of typing for optimization and safety checks at compile-time. SBCL is particularly great in that domain, employing a solid type inference engine.
- CL allows you to override optimization/safety levels at very small granularity - even sub-function level - with (declare (optimize ...)) forms - like, you can e.g. drop (declare (optimize (speed 3) (safety 0))) inside a loop inside a function, to optimize just this particular section of code.
- Even though CL type-related primitives aren't very convenient (they generate a bit of line noise in the code), the macro system gives you all the power you need to hide it under whatever syntactic sugar you like. There's nothing stopping you from writing (or finding a library defining) e.g. a macro:
(defun* foo ((real x) ((fixnum 0 100) y) z) -> rational
... some code ...)
(declaim (ftype (function (real (fixnum 0 100) t) rational) foo)
(defun foo (x y z)
(check-type x real)
(check-type y (fixnum 0 100))
(the rational (progn ... some code ...)))
--
I see "do the right thing" in the context of that epigram as meaning that you can choose to do the right thing without having to make compromises for syntax and semantics, as Lisp will happily let you remove any and all boilerplate with its macro system.
I would suggest, if you haven't already learned one, to find some FP language that looks interesting. Elm, perhaps, or Elixir. I'm not a fan of "hybrid" languages like Scala; finding one that hews closely to the FP ideas will be more useful, I think, in learning how to think functionally.
Any new language that doesn't operate exactly like the ones you already know can give you new ideas on how to program. Don't get bogged down in the expectations others have; just find one that piques your interest and dive in.
It can be functional, just as it can be object oriented or procedural, but those labels matter less to what Lisp is than does the intense focus on things like metaprogramming, in my opinion.
There was always more to Lisp (I read a cute essay from the mid-60s about how to balance assignment-and-goto style programming with recursive-pure-function style programming in Lisp), but older people making that connection isn't unreasonable, or younger people who've only heard older people talk about it.
More, early lisps were far more up front about their imperative abstractions. Something that we try our damnedest to hide from folks nowadays. In ways that are actually hard to fully explain. Used to, you were given an array of functions not just as a programmer, but as a user of the machine. The "side effects" of the functions were the point of them. They literally made the machine do something.
So, yes, functional has always been a defining element of lisp. I can fully support that statement. The defining element, though? I have a hard time supporting that one.
As for what makes Lisp "different," I think it was true in about 1960 that it was mostly the functional support. It hasn't been for a long time, but I just think that people who don't know Lisp assuming that it is isn't completely unfounded.
> More, early lisps were far more up front about their imperative abstractions.
I will say that I have no idea how someone could look at Common Lisp and not realise it was largely an imperative language unless they had some major preconceptions going in.
Love that quote from McCarthy, btw. I have not seen that before. Is there more in the context of where that came from?
So it might not be 60 years ago, but 40 is already quite some years.
Rather, I was claiming that most of the hallmarks of functional code in today's programs wasn't possible in older hardware. Specifically, many of the "functional data structures" that people are growing to love nowadays were decidedly not possible on so little memory.
I could also given Symbolics as example.
As 40 year old examples, in 20 years the use of Lisp improved a lot since the early IBM mainframes.
My assertion is that "functional" is not the defining feature of lisp. My evidence is much of modern functional programming was not done for a large part of it's history. My claim is further that many modern idioms couldn't be done on older hardware. Not that lisp couldn't run there, but that modern practices couldn't. Regardless of language.
Functional is certainly a feature. Even a prominent one. Just not a defining one.
I've never claimed that functional was not a facet of lisp. Just that it is not the facet.
Now, the claim I'm making that requires the most evidence is that many of the common functional datastructures people learn of today would not have been feasible on older hardware. I will not claim that they are too slow. I will claim that they are a bit too memory intensive for older hardware.
So, immutable lists are not equal to cons lists. Since most implementations allow direct modifications of cons lists. (Scheme bucked this trend, and people laud it for that. But it was certainly not the norm early on.)
Similarly, the highly branched vectors and other datastructures popular in clojure and friends are just too unfriendly to hardware. Could they have been done? Possibly, but the mutable structures had massive advantages on the hardware of the time. (Arguably, they still have advantages, just nowadays we get to make more tradeoffs between bleeding performance and maintenance.)
I'm definitely open to the idea what I'm saying is flat out wrong. I doubt I've reached my quota on stupid claims for my lifetime. And if my reading of your post was as off as one of the siblings where you weren't trying to contradict my claim, but strengthen it, my apologies. :) And my thanks for sticking with the thread.
Clojure simply differs too much from the Lisp languages. For example take into account the "atom" keyword and its meaning in Lisp versus Clojure. And also the way lists are used in Clojure, see "cons" for example.
These are not frivolous differences -- atoms and conses are the key building blocks of the Lisp language!
Plus parentheses rather than indentation to delimit expressions; parentheses are much more versatile once you get used to them.
This video was awesome, and entertaining as well, Mr. Tarballs-are-good/Symbol1cs/Stylewarning ("stylewarning" as a twitter ID made me ROFL). In particular, the comparison between calligraphy and Lisp programming was a very good one.
I've sent you a LinkedIn request, by the way. I'm the bald one in business suit.
I am very experienced and proficient with Python. Common Lisp goes way, way beyond what Python brings to the table. Take CLOS for example and compare it with Python's OOP facilities. CLOS is light years ahead.
Another differences (among many): Python is a high-level language. In CL, you can be high level and low level at the same time. For example, in CL you can disassemble your code to machine language and apply optimization directives and declarations to produce the shortest code, which can approach C speeds if done right.
That said, I still like Python a lot.
>Python is basically simplified Lisp.
... and I thank you for this phrase, perhaps I can use it whenever I need to justify my usage of Common Lisp at my workplace.
Thankfully we have now Julia, as yet another Algol-Lisp attempt.
In some cases, you can get around no-first-class-identifiers in Python by using strings and getattr(object, symbol) or locals()[symbol]. (What are other use cases of first-class identifiers, other than making some function arguments or macros look prettier?)
> Peter bravely repeated his claim that Python is a Lisp.
> Yes, John?" Peter said.
> I won't pretend to remember Lisp inventor John McCarthy's exact words which is odd because there were only about ten but he simply asked if Python could gracefully manipulate Python code as data.
> "No, John, it can't," said Peter and nothing more, graciously assenting to the professor's critique, and McCarthy said no more though Peter waited a moment to see if he would and in the silence a thousand words were said.
http://smuglispweeny.blogspot.pe/2008/02/ooh-ooh-my-turn-why...
You are completely missing the point. Not having identifiers be first-class objects with unique identity is how Python ended up being unable to reload code properly.
If you 'let people discover the truth' then you get a person who reads a couple chapters of SICP when they're 16, thinks they know Lisp, dismisses it as a cool idea language but not practical (compared to the more familiar and productive Python, Java, PHP..), and maybe just maybe ~10 years later they see an example like https://news.ycombinator.com/item?id=12222404 and say "I never knew Lisp, what have I missed out on?" But they could have been told what they were missing out on in the beginning! "Condescending Smug Lisp Weenies" may be Lisp Enemy #1, but I think Enemy #2, not far behind, is probably "People who think they know Lisp, but actually don't, write it off due to incorrect assumptions".
That said, I do think there's a certain something common to Lisp and Python that draws people to use both. e.g. Norvig. I think his page at https://norvig.com/python-lisp.html is a pretty balanced comparison.
Stanza feels like a Lisp with infix syntax. The closeness to Lisp explains why Stanza makes a semantic difference between "f(x)" (which is a function application) and "f (x)" which is a sequence of two elements (f and x). Unlike Nim which is strongly typed, Stanza allows to mix typed and untyped data freely.
[1] lbstanza.org
[2] nim-lang.org