Having never used Common Lisp, it's one of the languages I wish I've used for a long time.
However, working with Clojure and Scheme, I understand the power of repl driven development that is difficult to explain outside the experience of it.
However, working with Clojure and Scheme, I understand the power of repl driven development that is difficult to explain outside the experience of it.
The only lisp I've dabbled with was Racket and I found the repl driven development to be a frustrating necessity. I would spend a lot of time trying to figure out the actual type of the thing that I a given function (even a stdlib function) needed, and futzing around in the repl seemed to be the fastest way to do it, but it was still quite slow.
It reminded me a lot of my extensive experience with Python where things that are easy in, say, Go, are quite hard. People rave about their repl, but it feels like they're comparing it to a script iteration loop rather than static analysis tooling.
That said, I completely buy the argument that Python's REPL is particularly bad and that there are other use cases where REPL-driven development shines. My mind is open, but these are the sorts of experiences the pro-REPL folk should be prepared to engage with in their evangelism. :)
Common Lisp is remarkably more geared towards interactive use, but while it does make it easy most of the time to be used with just minimal REPL - as in literal (loop (print (eval (read)))) - it's best to use integrated, enhanced environments like SLIME/SLY/SLIMV/etc or one of the IDEs (LispWorks, AllegroCL, Clozure CL IDE)
You do know that statically typed languages have REPLs too? Like the ML family, including Haskell.
And when using something like a Jupyter notebook with a kernel for your compiled language https://github.com/gopherdata/gophernotes you can do similar interactive programming.
REPLs are about trying out ideas or changes.
Lisp REPLs take that a step further, as you interact with and in your whole actually running program.
With python, I can add a signal callback which calls the inbuilt breakpoint() function. This means I can send my application a given signal, let's say SIGTERM, and it'll drop into they python debugger whereby I can then drop further into the python REPL.
Does Lisp provide more than that out of interest?
Edit: ah I see hot reloading is a thing. Not explored that in python.
But Common Lisp also has a fancy system for recovering from exceptions. So after you replaced the broken function, you'd often see a list of "restart" points from where you could retry a failed operation.
So even if you failed in the middle of a complex operation, you could test out a fix without needing to abandon the entire operation in progress.
I'm not entirely certain if this was as useful as it sounds, in practice, but it was certainly very impressive.
I'm overgeneralizing (I've had remote repls in production python application and used them to hotfix things), but only a little. You will appreciate the difference if you work with long running computations, where you don't want to have to start over from scratch if something went wrong right at the end.
[1] https://lispcookbook.github.io/cl-cookbook/debugging.html
[2] https://lispcookbook.github.io/cl-cookbook/performance.html
I do, but that I don't see how that relates to the bit of my post which you've quoted. I certainly didn't claim or imply that REPL and static type systems were mutually exclusive, only that REPLs are a poor substitute for many static analysis tasks.
> And when using something like a Jupyter notebook with a kernel for your compiled language https://github.com/gopherdata/gophernotes you can do similar interactive programming.
Yeah, I'm aware. I operate a large JupyterHub cluster (among many other things) at work. :)
> Lisp REPLs take that a step further, as you interact with and in your whole actually running program.
That sounds nice, but it's too abstract to persuade IMHO. Personally, it seems like REPLs can make certain feedback loops a bit faster, and to that extent they seem like a modest quality of life improvement; however, considering how lisp folks positively rave about them, I assume I'm missing something.
Maybe I should have asked you what you use the REPLs of statically typed languages for.
I actually never used REPLs for that (neither in Python, CL, Clojurescript nor Elixir).
> That sounds nice, but it's too abstract to persuade IMHO.
Of course it is. I also didn't get it until I've tried it myself. That actually has been the main cause why I tried (Common) Lisp at all (well I had been using Emacs Lisp before, but that doesn't really count ;).
I don't really use them, which is kind of my point. Once in a great while I'll do some quick testing in the rare case that using a REPL is actually faster than just running the program, but to even call that a marginal improvement in my quality of life would risk overemphasis.
With a nice repl I can rapidly iterate trying different things and doing micro in-situ experiments to explore and understand how the language (or library more frequently) works.
It’s very edifying.
You are indeed, Rackets REPL is very barebones. Use Common Lisp with either SLime or Sly and Emacs.
How puzzling that you do not spell-out exactly what you believe they are missing.
That's just because your were a noob. You hadn't learned and internalized the standard library yet, and you hadn't learned to understand flow and structure of types and functions and variables.
I too when I started with Lisp had the same struggles, but they go away with expertise.
That's why you'll hear people talk about "training wheels", when you program in a statically typed language with an extensive IDE doing a lot of static analysis for you, you basically operate constantly with training wheels on. You never really learn what class does what, what methods they have, what types things are, you rely heavily on your IDE.
For example, ask an experienced Java developer to write some Java in notepad? They don't know what to import, where to find anything, completely lost, productivity hits zero.
And I'm not degrading Java devs or that setup. What happens in Lisp is that you have to learn your language, you can't rely on an IDE or the compiler. In theory you could also learn Java to that level, stop using an IDE and you'll be forced to really know the language as well.
Here's the kicker? Once you learn the language to that level, your productivity in it skyrockets! But until you do, its a slow crawl.
And it's only ounce you've got that experience that the ability to change the program as it is running through a REPL really brings about speed, you don't use it to figure basic things like types and what standard functions exists, but to tackle real business logic or explore libraries and 3rd party APIs to get a grasp on those quickly.
Take this from someone who went from strongly typed Java, C++, C#, ActionScript 3 to a Lisp. I'm now way more productive in Lisp.
P.S.: Also, you'll have static analysis in Lisp as well, depending on which one and your setup. In Clojure, I have static analysis, an IDE, a connected REPL, and use all three actively.
An IDE (for a sufficiently static language) eliminates so much unnecessary mental load that is otherwise wasted on avoiding inevitable human errors—typos, type mismatches, forgotten cases.
I don’t want what is essentially trivia swimming around in my head any more than I want to find out about the inevitable errors I’ve made when I come to compile or run my program.
I want to know that no such issues exist well before then, so I can focus on making sure my program actually behaves correctly.
And I don't mean to say that you're a bad developer or not smart enough if you use an IDE. Of course you should use an IDE with Java or C# or C++. Those languages are all very good, and when paired with a good IDE can be quite productive. And even with a Lisp, you should and will want to use a powerful editor or IDE with good auto-complete, linting, jump to definitions, etc.
What I'm trying to say is that to be productive in a Lisp, it's a different learning curve.
You just can't come and say: "well I tried it a little bit, and I couldn't figure out how to do anything and I was really slow, so I just don't get the point of a REPL and I don't get why people claim it's a hugely productive tool."
I'm telling you why, it's like complaining that you type slower on a keyboard than you can write by hand for the first year of you using a keyboard. There's a learning curve.
You can't just say: "well I can't figure this out. Where is the IDE to help me with it? Where are the type definitions? Where is the compiler to guide me? I guess the Lispers are just full of it."
No, you didn't do what Lispers do. Lispers learn the language until those things are no longer a problem, and then they unlock a new found productivity combined with the ability to work inside a running program, quick iteration, being able to extend and mold the language to what is most efficient for you and your problem, etc.
It's 100% ok if you say that you don't feel it's necessary for you to go the extra mile to learn a Lisp properly. That's fine, if you find you're sufficiently successful and productive already with your language, that's totally cool. But you can't not properly learn it, and then claim that the claims of those who have are invalid.
It's like if you started to doubt a touch typist that's telling you that learning to touch type makes you a more productive typist, because when you tried it was hard to learn and you were slower at typing.
I was a teenage Lisp enthusiast and have seen these sorts of Lisp discussions go round and round in circles on online forums for the past couple of decades. Ultimately, the corpus of publicly available software written in Lisp just isn't that compelling.
It's often said (and may be true, for all I know) that there are lots of companies quietly using Lisp behind the scenes and reaping the benefits without shouting about it. But to be persuaded, people need to see results of the claimed productivity gains in the form of actually existing software.
It's also worth bearing in mind that there are lots of people (like me) who did give Lisp a serious try (Common Lisp in my case) and then moved on for various quite sensible reasons. Lisp enthusiasts sometimes speak as if anyone who doesn't love Lisp can't possibly have given it a thorough trying out, but I would question that assumption.
As for example of software made with a Lisp, I feel people are never happy with them, like a constantly moving goal post, but there's a certain chicken and egg to it.
Some examples:
- HackerNews
- Emacs
- LispMachines (the entire OS)
- AutoCad (uses Lips as a scripting language)
- Jak and Daxter (the first game)
- Yahoo Stores (pre-sale)
- Reddit (pre-IPO)
- Datomic (no-sql tuple DB)
- The flight system in the Boeing 747 Next (their latest 747 as of this writing)
- Apache Storm (the distributed stream processor)
- Walmart (they use it for backend stuff)
- CircleCI (build automation)
- Riemann (a high scale distributed event stream publisher)
- worldsingles network (bunch of dating websites)
- NuBank (a Brazilian bank)
- ConsumerReports (all their backend)
- The Climate Corp (agriculture software)
- Roam Research (note taking app)
- Kevel (ad server as a service)
- HighSpot (sales rep software)
- Reify Health (health related software)
There's also a lot of use of Lisp that's not like a closed software, a lot of times it'll be a backend service or parts of a system is using Lisp, I know Amazon, Microsoft and Netflix have some such use, it's not wide scale, but you'll find some teams there using some kind of Lisp, as you'll sometimes see job offers that mention it.
Something like C's K&R or Lua ones?? Its sooo handy to have a clear picture from the creators themselves.
If you are saying "memorise what you commonly use" then that's fine, except that any typical program usually involves a lot of "lesser used" stuff as well.
How? As in what commands/functions/macros?
Say I define a function and compile with slime, how do I get the repl to print it back to me? And with a function I loaded from an external library? Is there something like macroexpand-1 for functions which will print it out?
Common Lisp also has built in the function FUNCTION-LAMBDA-EXPRESSION, which returns the source for a function -> when Lisp has recorded its definition.
Try (DESCRIBE #'some-function) in SBCL to get information about the arglist, the types, the source file.
In a SLIME editor buffer try c-h m . This will display the buffer commands. There are a zillion of commands to get information about the symbols, code references, etc.
On my Lisp Machine:
Command: Show Source Code (a defined function spec) mts::init-facts
Source code for MTS::INIT-FACTS:
(defun init-facts ()
(setf *init-facts*
'((world (loc (actor joe) (val cave)))
(joe (loc (actor joe) (val cave)))
(world (loc (actor irving) (val oak-tree)))
(irving (loc (actor irving) (val oak-tree)))
(joe (loc (actor irving) (val oak-tree)))
(world (loc (actor water) (val river)))
(joe (loc (actor water) (val river)))
(world (loc (actor honey) (val elm-tree)))
(irving (loc (actor honey) (val elm-tree)))
(world (loc (actor worm) (val ground)))
(joe (loc (actor worm) (val ground)))
(irving (loc (actor joe) (val cave)))
(world (loc (actor fish) (val river)))
(irving (loc (actor fish) (val river))))))
Command: Show Callers (a symbol [default MTS::INIT-FACTS]) MTS::INIT-FACTS
MTS::MICRO-TALESPIN calls MTS::INIT-FACTS as a function.
MTS::MICRO-TALESPIN-DEMO calls MTS::INIT-FACTS as a function.
Done.
Command:As opposed to say learning the standard library relating to MIDI access, or file IO, or other specialized things that in theory could be a library and aren't really core.
It might sound a little like syntax, but because Lisps have such a lean syntax, there's a lot of the fundamental constructs that are part of the standard library and they are more what I'm referring too.
For example you'll have things like: car, cdr, append, mapcar, some, every, symbols, keywords, loop, intern, quote, format, butlast, intersection, union, multiple-value-bind, apply, funcall, =, eq, eql, if, unless, typecase, cond, dolist, iterate, etc.
What often happens in Lisps is that each core function is very powerful and becomes almost a mini-DSL. So it takes a while to really learn about all those and have them memorized by heart so you don't constantly need to refer to the documentation about what they do and how they work and how to use them. And often their power comes when they are combined together as well, and that also takes some time to learn all the valid ways of combining them.
Before you are familiar with all that and it's second nature, almost instinctive, you'll be a lot slower to read and write code.
Can you give some examples? This is not something I've encountered (although I have much more experience with Python than with Go so there is probably plenty in Go that I have not encountered).
> the argument that Python's REPL is particularly bad
What argument is this? This is also not something I've encountered. I use Python's REPL all the time and it greatly speeds up development for me.
Note that I'm not a 10x master, I use CL for mundane, real-world tasks (access to an FTP, parsing XML, DB access, showing products on a website, sending email with sendgrid…), my programs had 1 trivial bug, I am more productive and I have more fun… Just to say it has its uses, even today, even with "all the libs" of Python (and btw, see py4cl).
I'm not sure I understand. When I run Python's REPL, I'm running the interpreter on my local machine; I'm not talking to any server. The same would be true if I run the Lisp interpreter, but I don't see how that is any advantage for Lisp over Python.
When I've done Common Lisp heavily, I've lived in the same instance between multiple days up to weeks. This feels perfectly natural.
Admittedly it is hard to justify why this difference is meaningful.
Anyways, do you develop your app when you are in the REPL? Or do you try things out in the REPL and write back what you found out in the files? If you found how to send changes from your editor to the REPL, then good, but it's not built-in, and it will lack the other things I mentioned.
Yes. :-) Both strategies can be useful. (And note that "app" does not necessarily mean "web app"; web applications are just one kind of application. There are lots of different kinds of applications. The REPL helps with all of them.)
> we compile our code function-by-function with a keystroke and we get instant warnings and errors, when there is an error we get an interactive debugger that pops up and allows us to inspect the stack, go to the buggy line, recompile, come back to the REPL, resume the execution from the point we want, objects are updated when a class definition changes and we can even control that.
I know you are skeptical because python has a repl too and so it is hard to see what these "live" environments like lisp/smalltalk are offering. But trust me (and the others) that there is more than what python offers.
I have written python professionally probably more than any other language I have used, so I'm not ignorant of python btw.
No one is trying to fool you. But you don't have to take our word, please try it out for yourself!
I understand it just fine. I've used a Lisp REPL. I'm just pointing out that not all of the features of the Lisp REPL that are being mentioned are unique to Lisp.
> I know you are skeptical
I'm not "skeptical" at all about what can be done in a Lisp REPL. As I just said, I've used it. I have not claimed that a single thing anyone has said about the Lisp REPL is false.
I think you are reading things into my posts that I have not said.
> you don't have to take our word, please try it out for yourself
As I said above, I already have. Don't jump to conclusions.
This looks like something specific to your particular development workflow for one particular type of application (a web server). REPLs (for both Lisp and Python) are much, much, much more general than that.
Specifically, if you try to paste a function definition that has blank lines in it, into the REPL, the first blank line will be interpreted as "end of function" and the rest of the function will become incorrectly indented garbage.
I've been having trouble with this as well. I think we'd do best in removing "REPL" from any explainations, and instead use the words "Interactive Development", "Dynamic Development", "Hot Reload on Steroids" or something similar, as when I use terms like that, people understand it's nothing like the "repls" ("shells" really) other languages provides.
The first Lisp, called LISP I and its successors, from John McCarthy and his team had a remarkable collection of innovative technology:
* a reader for symbolic expressions. These are nested lists of symbols and other data objects like numbers and strings
* a printer for symbolic expressions.
* an evaluator for symbolic expressions. Each symbolic expression evaluates to a value. Side effects to the running Lisp could be added variables, data or function definitions.
* an automatic garbage collector taking care of freeing no longer used memory
* piece-by-piece allocation of data
* a form of programming with functions, also using recursion
* a way for Lisp to store and restore the main memory heap to/from external storage
* Lisp compiler written in Lisp, which generates assembler, and an assembler which creates code into the runtime -> the first interactive incremental to memory compiler
* a way to program with Lisp code generated by Lisp macros
It had a batch REPL. This Read-Eval-Print-Loop would read a file of expressions, expression by expression, evaluate them and print each return value into a file. It used the above functions READ, EVAL and PRINT.
Lisp was starting up and restoring a default image as the heap. It would then execute the file via the batch REPL and save a new, changed image as the new default image. Next time Lisp starts, it would then restore the latest image.
A next step in the evolution was then to connect a terminal and let the user enter a Lisp expression. Lisp reads it, evaluates it and prints the value back. Side effects again might change the running Lisp. The REPL then waits for the next terminal input. Code could also be loaded from files.
That way the REPL was not some interface which the debugger brings up on request. It was the main interface for the programmer to build up large Lisp systems piece by piece.
We are talking about the early 60s here.
In the coming decades the idea of a REPL morphed into extensive interactive resident development environments like Interlisp <http://interlisp.org>. Interlisp combined the REPL with managed source code, structure editing for code and data, automatic error correction of code, undo of evaluation effects, a virtual machine, ... and a lot more. Here Lisp, the IDE and the programs under development were running in one Lisp runtime. We are talking about the early 70s.
Interlisp moved at Xerox PARC onto their new graphical workstations. Thus the REPL-based development environment moved to graphical user interfaces. The Interlisp team got an ACM Award for their pioneering work.
The MIT AI Lab took that idea to their graphical Lisp Machines.
Today in 2021 Lisp programmers often use SBCL as the implementation, GNU Emacs and SLIME (or similar) as their development environment. It's still usual to develop a running program, having potential multiple REPLs into the program, using Emacs/SLIME to send code for remote evaluation to the running Lisp.
GNU Emacs itself is a Lisp program, too - with its own integrated Lisp and the development tools for it. GNU Emacs has its own REPL, called IELM, the interaction mode for Emacs Lisp.
Even in Lisp not all development interactions goes through a REPL, but there are still a few very powerful ones, like those of Allegro CL, LispWorks, McCLIM, Interlisp (recently open sourced) or the Lisp Listener from Symbolics Genera. Typical is the rich interactive error handling in multilevel break loops with restarts and the ability to continue from errors. The above also support image-based development and integrate Lisp, IDE and the software under development in one program.
Is this not the same?
I haven't used Common Lisp much, but I've seen a few really impressive demos of debugging in its available REPLs. This is a longer one, but a good tour of SLIME (also an Emacs package): https://youtu.be/_B_4vhsmRRI
Here is a video I recorded which hopefully demonstrates just how much faster you can go in Clojure
Like I think a parent comment said it's hard to explain but once you get it, it's fun malable and exploratory
Intellji also has a debugger that works with Clojure for when it's appropriate
In php you can't redefine a function at runtime without something like runkit, in Clojure you can fix your systems at runtime
Common Lisp takes this to another level though with stuff like restarts. Where the app will throw an exception during development (perhaps because a function isn't implemented yet) and instead of crashing you'll end up in the repl where you can make some fixes and keep running the app.
An even more impressive instance of remote debugging occurred on NASA's 1998 Deep Space 1 mission. A half year after the space craft launched, a bit of Lisp code was going to control the spacecraft for two days while conducting a sequence of experiments. Unfortunately, a subtle race condition in the code had escaped detection during ground testing and was already in space. When the bug manifested in the wild--100 million miles away from Earth--the team was able to diagnose and fix the running code, allowing the experiments to complete. One of the programmers described it as follows:
Debugging a program running on a $100M piece of hardware that is 100 million miles away is an interesting experience. Having a read-eval-print loop running on the spacecraft proved invaluable in finding and fixing the problem.
That looks like about 9 light minutes away, so about 18 minute latency?
It might be super epic, but doesn't sound super nice :)
I.e. in each notebook cell, prototype and exercise your function in isolation. Then copy them over the real py module.
Still, I miss the elegant data literals in Clojure whenever I'm slogging through some literal Dataclass objects for testing..
# %%
The only reason to do a full restart would be if you changed the data structure expected by the functions in a way that's not semantically backwards compatible.
Big-whoop on a stateless webserver, maybe, but on the front end using ClojureScript for an SPA it is nothing short of magical and has ruined any other browser dev work flow for me.
And when control reaches a function that expects the old kind of object and runs into an error, it'll enter a breakloop that you can use to modify the now-out-of-date function to handle the new definition properly. Once you've redefined the function you can then tell the runtime to resume and it will proceed as if the function you called had originally had the new definition.
https://mikelevins.github.io/posts/2020-12-18-repl-driven/
https://mikelevins.github.io/posts/2020-02-03-programming-as...
not even close, no