Show HN: Full Stack Lisp – A book in progress about writing Common Lisp apps
fullstacklisp.com
fullstacklisp.com
Digital books are so broken!
If you scan a dead tree book to PDF, and you include an image of the outside, that's still called a cover; PDF isn't broken.
If that isn't broken then what is?
Why, viewing content on some god forsaken web site, page by page, through some proprietary plugin, with no ability to save other than taking a screenshot of the window.
Downloading and saving is king.
I like how you mention first thinking Common Lisp was archaic, and then realising how advanced it really is.
BTW, you might want to wrap your text lines:-)
"You know Lisp is the perfect language precisely because it lost its momentum (died) and lives on. Other languages need their Next Big Thing momentum."
http://smuglispweeny.blogspot.com/2013/08/wow-even-lispers-h...
Common Lisp was exactly what it says "Common". Just some sort of common ground between all the various lisp dialects that existed, because everyone went their own way in extending the language. People end up coming back to it time and time again, because all the others don't quite provide that common ground for people to work on. Whilst there may be things missing, there's always something more missing in other lisps.
I still think Shen has lots of potential.
It is especially good to see a complete caveman2 application developed in the book.
Yes, Lisp seems to me to be getting more exposure, but that may be my 'Lisp-colored sunglasses' ;)
Is caveman2 a lot easier, or what other advantages does it have over piecing stuff together?
I have always liked and programmed in Lisp, but not for the Web, so I need to get up to speed with Web frontend/backend programming again for personal use, not a job.
OT: Mark, I am on your 3 edition of 'Loving Common Lisp', and I really appreciate the Networking chapter update, and your smooth writing style!
As for comparisons, this is a Gorilla vs Shark problem (https://blog.stackoverflow.com/2011/08/gorilla-vs-shark/), without context you can choose anything you prefer.
LFE is not really Lisp, it is more like Erlang with parenthesis, so it is hardly comparable. Erlang offers a very specific, and useful, model of computation: if you need that, use Erlang or LTE.
In CL vs. Clojure, I suppose CL looks riskier to use than Clojure, w.r.t. all those JVM libraries (and the coolness factor). I think this is false, because you can access Java/Clojure/C libraries from CL with ABCL/CFFI, whereas you cannot access the Common Lisp ecosystem from Clojure easily.
But in the end, which programming language you use depends on many factors.
Is it good at number crunching (which is not Clojure's and definitely not Erlang's forte)? Is it good at low level systems programming? Is it good for computers with low memory resources? Etc. In your words, I am looking for the context.
When I switched from Clojure to Common lisp a few years ago, mostly I really like using multi-paradigm languages. Clojure code wants to be functional. Common lisp code has no such preference, and with the excellent OOP support it's a pleasure to use. Basically it's generic function based, rather than class based, which means that your methods are owned by your generic functions, rather than your classes. This gives you much more flexibility and power than "standard" OOP. I can write software in the style that best supports the problem I'm trying to solve. Other great features include the condition system, basically exceptions on steroids. Very few languages have something similar.
Another one of my favorite features is that the standard has been stable for decades. The core language hasn't changed since 1994, so the ecosystem continues to evolve constantly, but the core stays stable and it isn't unheard of for code written in the 80's to run with little or no modifications. I like the idea that my code might have that kind of longevity.
I could go on, but you get the general idea of having an extremely mature and stable language with many implementations that all try to solve different problems, with sufficient portability that fragmentation doesn't tend to happen(like in the scheme world for example). And personal taste of course, It's just a pleasure to use for some reason. I don't know why, but I'm just having way too much fun playing with it and trying to explain it to others.
Common Lisp and some older Lisps, along with Smalltalk and FORTH, treat programming as an ongoing interaction in which your main program is already running before you ever write any code. It's running, but it doesn't yet know how to be your application. You have to teach it. This you do incrementally, one small change at a time, gradually guiding and shaping the running program until, eventually, it becomes your application. It continues to run throughout the entire process; it just changes its behavior as you teach it new capabilities.
Common Lisp (and a few other languages) are designed with this approach to programming squarely in mind. Consider, for example, the standard Common Lisp functions CHANGE-CLASS and UPDATE-INSTANCE-FOR-REDEFINED-CLASS. In most languages these functions make very little sense. Once you understand how Common Lisp is designed to be used, it's obvious that they are necessary. They're necessary because obviously you're going to redefine classes as you progress, and obviously you don't want your running application to fail simply because you've redefined its types out from under it. So the language is designed to support that way of working. It provides facilities for gracefully handling the situation that arises when you redefine types out from under your code, and it gives you ways to ensure that the program keeps running as you make those changes.
This way of working is now sufficiently out of fashion that even new Lisps don't necessarily support it. Clojure isn't great in this area. Neither are some Schemes. Common Lisp and Smalltalk are the strongest remaining bastions of programming-as-teaching, as far as I can see.
Almost everywhere else in programming today, tools and languages uncritically assume the programming-as-construction model. Maybe that's okay. There's certainly good work being done in that mode. Programming-as-teaching is my native idiom, though. I'd guess that I'm somewhere around ten times more productive when working that way. That doesn't mean it's a better way, of course; it just means it's better for me. Nevertheless, I can't help but mourn when I think that maybe programming-as-teaching is disappearing from the world.
What is it about CL / Lisp that makes people think it more suitable than other languages for using in such a style? It cannot be only the REPL, since others have them too. Or are CL REPLs more powerful? I've only explored one a bit, Franz, IIRC, some time ago. Edit: Okay, you gave two examples above, CHANGE-CLASS and another. Any other good ones?
Also can you elaborate with some examples on how the condition system is more powerful than other error handling approaches? I saw that mentioned here on HN recently, IIRC.
Common Lisp isn't the only language designed that way. Older Lisps were, too. So was Smalltalk. But that view of programming is now so far out of the mainstream that even new Lisp dialects don't reflect it.
But you asked for examples of other Common Lisp functions that take interactive programming into account. Instead I'll offer a mixed list of functions, special forms and design features. Don't assume it's exhaustive; there are too many to reasonably expect that I can list them all.
- COMPILE doesn't mean compile a file; it means compile an arbitrarily chosen function that might not even be in a file anywhere
- EVAL-WHEN enables you to control whether forms are evaluated at compile time, load time, or toplevel-evaluation time, because you might have program text that is intended to be evaluated at any combination of those times. Again, its behavior is independent of whether the affected forms appear in files (though obviously that's the most common place for it to appear).
- namespaces are represented by packages, which are runtime objects that you can interactive inspect and modify. The namespaces represented by packages have a concept of public and private names, but you can't really hide private names, because if you could it would be impossible to change them at runtime. Common Lisp is allergic to features that are impossible to change at runtime. It has to be in order to support interactive programming.
- CLOS has a whole protocol for changing the definitions of classes and functions while a program runs. CHANGE-CLASS and UPDATE-INSTANCE-FOR-REDEFINED-CLASS are members of that protocol. SO are REINITIALIZE-INSTANCE and SHARED-INITIALIZE and UPDATE-INSTANCE-FOR-DIFFERENT-CLASS and ADD-METHOD and REMOVE-METHOD and DEFINE-METHOD-COMBINATION.
- introspection is pervasive in Common Lisp. It has to be. It's an interactive system in which you build programs while they are running. You have to be able to examine every part of the running program as it runs. So it has scads of functions for examining the structure and value of everything in the running program--and not just examining it, but changing it. There's BOUNDP and SLOT-BOUND-P and SLOT-EXISTS-P and FMAKUNBOUND and FIND-METHOD and TYPE-OF and CLASS-OF and SYMBOL-FUNCTION and so on and so forth. DISASSEMBLE, which decompiles compiled code so that you can interactively examine what the compiler produces, is part of the language standard.
- the assumption of interaction is also pervasive. TRACE and INSPET are part of the standard. So is BREAK--a function that causes the program to enter an interactive breakloop in which you can examine the dynamic state of the suspended computation, inspect and change the values of local variables, redefine functions and methods, and resume the interrupted computation when you think you've made the changes you wanted.
- you asked about the condition system. You could say that it's Common Lisp's implementation of exceptions, and that would be true as far as it goes, but it would also ignore important aspects of its design. Yes, you can raise a condition, just as you can raise an exception. However, the normal behavior of an unhandled condition is to drop you into an interactive breakloop from which you can modify the suspended computation and all of its accessible state.Conditions provide restarts -- a mechanism for offering a set of options for resuming control at a place of your choosing, or of selecting how to resume execution under program control. Conditions are also more generally useful as ways to manage nonlocal transfers of control.
- it's not explicitly part of the standard, presumably because it was such a fundamental assumption of Lisp programmers when Common Lisp was being defined, but nearly all Common Lisp implementations support saving a serialized form of the dynamic run-state of the system that can later be reloaded to reproduce that dynamic run-state exactly. It means that you can save the exact environment that you're working in right now, complete with interactive state like breakloops and so forth, and resume it later. You can, for example, discover a bogu, dig into what you can find out by inspecting the dynamic state of a breakloop triggered by the bug, and save off that whole dynamic state, then reload it on another machine so that you can show it to a colleague in all its glory.
There's more, of course. It's hard to reduce a design that is all of a piece to a list of discreet features.
And yeah, it's sad that "programming-as-teaching" fades out of fashion. Now that I know what it's like, working in an environment that doesn't support it always leaves a bitter taste. Let's just hope more people try it, because it's addictive.
These systems are so coherent they solve themselves in and out, but if you mess a few things it's now compounded interests fueling your technological debt.
I've been dabbling in Clojure for about 4 years now, off and on, and it works very well for small-ish projects. The integration, as others have mentioned, with the JVM is superbly seamless (it leaves Scala in the dust in this respect) and this opens up most of the Java ecosystem. Also, the STM model of concurrency management is super cool. So far so great!
But every time I try to build something more substantial than a Project Euler submission, I end up with something blowing up in my face. Recently, I was a bit annoyed to discover that the set of keys of a map doesn't have the same semantics (including sequential access) as other sets, and on another occasion I was debugging something and the contents of some variables that displayed valid data in "print" statements showed as "null" in my debugger. None of this is show stopping but when I'm just playing around I have a low frustration threshold. My impression is that Clojure is a good Lisp with many fantastic new features but not all of the batteries are included (yet).
Meanwhile, I have zero experience with CL but I keep hearing that it's highly mature, rock solid, the choice of most professional Lispers, and the platform of choice for Peter Norvig's book on AI programming. In my imagination, that means everything works exactly as expected and there is a wealth of useful libraries - which would be sweet, from where I stand.
Uh, what?
It is definitely highly mature, rock solid and the choice of most professional Lispers. It anticipates many problems one doesn't even know one will have, and solves them.
In the negative column, it's a bit crufty for historical reasons (e.g. (aref ARRAY 1 2) vice (gethash KEY HASH-TABLE)) and made some choices which in retrospect were foolish (upcasing symbols being a huge one). The former doesn't really matter, and the latter can be solved with a simple (setf print-case :downcase).
For a long time, there were few good resources for learning Lisp. Fortunately, that's changed. Peter Seibel's Practical Common Lisp (http://www.gigamonkeys.com/book/) is a must-read: it explains a little of why Lisp is so relevant for modern development. Edi Weitz's new book Common Lisp Recipes (http://weitz.de/cl-recipes/) is worth its weight in gold.
Quicklisp (https://www.quicklisp.org/beta/) is a great place to find a lot of great packages.
(Tdd is writing some tests then making some code that passes them. Which, imo is cart before the horsism)
For more examples, see http://lisper.in/reader-macros/.
It is a way to customize the Lisp reader by associating a custom function to an entry in the readtable.
So you can say that character '[' will be tied to a custom reader function, which reads from a stream and produces a Lisp object, at readtime.
The standard sharpsign[1] character is used to read different kind of structures: complex numbers #C(0 1), structs #S(tree :left nil :right nil), and so on.
In particular, you can have sharpsign-dot, which reads a lisp form, evaluates it at read time; the result is the form being read.
Also, you can use #1= and #1# to affect parts of your code to reader variables and reuse them in other places.
If the PRINT-CIRCLE (there are earmuffs around it, but I can't format it here) variable is set to true, here is a quine:
#1=(print '#1#)
[1] http://www.lispworks.com/documentation/HyperSpec/Body/02_dh....
Hyperspec reference for predefined sharpsign reader macros: http://www.lispworks.com/documentation/HyperSpec/Body/02_dh....
A blog post where Vsevolod Dyomkin creates a reader macro to notate list comprehensions to his liking: http://lisp-univ-etc.blogspot.com/2013/01/real-list-comprehe...
A blog post where Inaimathi mentions using CLSQL reader macros to facilitate reading SQL expressions in code: http://langnostic.blogspot.com/2011/03/clsql-and-nothing-els...
Let Over Lambda chapter 4 which has various examples: http://letoverlambda.com/index.cl/guest/chap4.html
The best bit though, is that this isn't some nasty hack. This is supported by the spec, so I can package this up and let other people us it just like any other functionality we care to ship around.
That's my favorite thing really, being able to treat approaches to writing code in the same way we treat the functionality we make using code.
You mean ... programming the programming language?
> I can package this up ...
Not really; it will clash with someone else's use of λ. Read macros do not "package" particularly well. (Racketlang has a good solution for this: you put a #lang whatever directive at the top of a file and then the rest of the file uses the special syntax associated with that lang name).
It's very easy to overestimate the usefulness of read syntax.
This feature of Common Lisp is used far less than newcomers might imagine, though there are some famous examples of it. If you randomly sample Lisp code, you are much more likely to find a macro than a read macro.
> So within the hour I had something that let me write λ( _ _)*
Do you plan to type "lambda" with args enough times to eventually that hour of your life back? Ha.
Useful syntax for writing anonymous functions can be had, without relying on read macros at all.
cl-op: https://code.google.com/archive/p/cl-op/
Note how if we take λ(* _ _) and then just move the parenthesis over the Greek symbol, we get (λ * _ _). This saves almost the same amount of typing. If your Lisp recognizes λ as a symbol token, then all you have to do now is write a regular macro. Basically, just get the cl-op package and then set up λ as an alias for op: (defmacro λ (&rest args) `(op ,@args)).
In TXR Lisp, I implemented a more powerful op which has numbered arguments, and an escape syntax for inner ops to refer to outer ops.
Y combinator with TXR Lisp op:
;; The Y combinator:
(defun y (f)
[(op @1 @1)
(op f (op [@@1 @@1]))])
;; The Y-combinator-based factorial:
(defun fac (f)
(do if (zerop @1)
1
(* @1 [f (- @1 1)])))
;; Test:
(format t "~s\n" [[y fac] 4])
(defun dup (fun)
(lambda (arg) (funcall fun arg arg)))
(funcall (dup #'*) 2) -> 4
(mapcar (dup #'*) '(1 2 3)) -> (1 4 9)Documentation for merge-readtables-into:
"If a macro character appears in more than one of the readtables, i.e. if a conflict is discovered during the merge, an error of type READER-MACRO-CONFLICT is signaled."
At their core, reader-macros are abbreviations. Obviously there are a finite number of abbreviations one can use, so any time you want to add concise syntax from more than one source, there is a chance of conflict.
> Not really; it will clash
Nope, named-readtables fixed the clashing reader-macro issue a good while back.
> You mean ... programming the programming language?
Yes, I do. I said it the way I did out of choice, as the 'programmable programming language' phrase has been bandied around so often that, for some, it has lost impact. Maybe that phrase doesn't inspire the same ahah moment in some as it does for other. Saying the same thing in a different way can be enough for someone to pick up something new. (It feels weird writing this down as it's kind of self evident)
> Do you plan to type "lambda" with args enough times to eventually that hour of your life back
I that hour was spent learning about reader macros, learning something new is of value to me, so there is no 'lost hour' to reclaim. Don't you just code for fun some days, without worrying if it is somehow ultimately useful?
> Note how if we take λ(* _ _) and then just move the parenthesis over the Greek symbol, we get (λ * _ _). This saves almost the same amount of typing
Ah ok, maybe this is where we parted mental ways. I'm not interesting in saving typing, that wasn't the goal. The point was to remove a little visual obstruction from what I was doing. For example (and yes it is a trivial example) (lambda (x) (* x x)) is 20 characters long, of which 8 are of interest (* x x) I liked that the shorthand reduced a little visual noise.
Yes there are many ways to skin this particular cat, but I liked the shorthand. I also like not having the lambda in the first position as it maintained a property of lisp code I generally enjoy, that I can run my eyes over the lefthand side of the forms and quickly get a rough feel of what is going on. Again, this is just a personal feeling, but some days I write code just for me, for fun, so why not make things pleasant for myself.
It's rather odd to get into a discussion about this, when the whole point is that we don't have to pick one golden way and enshrine it in the language. It's just a library, use it or don't, it doesn't matter.
Peace
If it's written in a functional style it becomes even easier: you don't have to hold any state in your head.
- Line numbering is a little too close to code
- There seem to be two kind of fonts for code, some render bigger than the other on two different devices
- Texboxes are too narrow and as a result the code wraps around a lot.
Maybe this due to my settings. The result might be different in the PDF.
If it is the latter, that would be quite compelling. Clojure/Clojurescript is quite compelling from a developer point of view, since you don't have to switch mental context to develop your server and client code (as you would have to do if you develop Go on the server and JS in the browser).
I've been seeing (common) lisp everywhere lately, from talks on clasp to the recent(ish) work on (really) fast http parser for lisp etc. It really feels like cl, with the help of a few, but very much needed tools and libraries (like quicklisp) and mature implementations that compile down to machine code could have a bit of a renaissance.
I think a lot of people simultaneously realized that one didn't have to code old style lisp, just because the cl spec has been written. Like the project to reimplement cl in cl. There's no need to rely on old broken crutches, just take the good parts of cl, combine with 40 years of experience from system and language engineering, and reduce back down to cl. And suddenly you have something very elegant and efficient and powerful.
Ok, I admit, I still want "sweet expressions"[1] -- but I think just a simple thing like acknowledging that global variables with * earmuffs * , are still global variables - and generally a bad idea -- and that cl is much more than some small "functional" language goes a long way. I remember I came to lisp rather late, and was confused by how un-functional a lot of the code was. There was just a lot of messy examples with detached functions for manipulating ad-hoc data structures that didn't seem properly motivated at all. And it all seemed terribly brittle.
But if one ignores all of that (and a lot of the classic lisp books), what emerges is that the "good" parts of cl are really great, and there's really no reason not to use those parts. And the warts in moderation (as with any other language).
There's a Common Lisp library on Sourceforge: http://readable.sourceforge.net/
I've been thinking of using it to make a Lisp interface palatable to non-Lispers. We'll see though.
While I have to say that it has worked pretty well, Lisp and Clojurescript are different enough that it really isn't much of a benefit to use both together. Even the S-expression format of the two languages are so different that we need to use JSON when sending messages from the server to the client (and vice versa, of course).
Please feel free to look at the code if you want to learn more about how it's done. I'll be happy to answer any questions: https://github.com/cicakhq/potato
With this book, we want to show you how to use functional programming and Haskell to solve realistic problems. This is a hands-on book: every chapter contains dozens of code samples, and many contain complete applications.
Haskell has a discoverability problem, and a good modern book might be the best way to solve it. But it may be moving too fast for that currently.
Ask me how long I spent banging my head against the wall trying to get OpenGL working in CMUCL with Alien. Jesus jumping-jack Christ. I can only hope things have improved in the past 12 years or so. But I highly doubt it.
So where are the:
- AOT compilers to native code
- REPL environments with GUI debuggers, with fix-and-continue?
- Ability to do systems programming without having to use an external language?
- Macros
We gave you the benefit of the doubt originally, despite the nasty embers burning in your comment, but then you went full flamewar downthread. We don't want that on HN, so please don't do it again.
How about fitting, into a single image, a joke about some ex-web developer, stacking pancakes at IHOP.
Or maybe a visual reference to this type of full stack: https://www.google.com/search?q=guitar+full+stack&tbm=isch