Why I haven't jumped ship from Common Lisp to Racket just yet
fare.livejournal.com
fare.livejournal.com
Also, related to that point: There are many different CL implementations out there that satisfy different use cases, like for example JVM deployment (ABCL), embedded systems (ECL), speed(SBCL), fast compile times (Clozure), pro-level support (LispWorks, ACL), etc. So the same code has a huge amount of options for deployment. It really makes Common Lisp be "write once, run anywhere".
Then speed is also never mentioned. Lisp can be seriously fast; under SBCL it is generally 0.3x to 1x C speed; LispWorks might be faster, and there's a PDF out there called "How to make Lisp go faster than C", so that should give an idea of Lisp speed potential.
CL wasn't created by philosophing about what a programming language should be for many years; CL was basically created by merging Lisps that were already proven in the industry (Maclisp, Zetalisp, etc), already proven to be good for AI, heavy computation, symbolic computation, launching rockets, writing full operating systems, etc.
CL is a "you want it, you got it" programming language. You want to circumvent the garbage collector? need to use GOTOs for a particular function? want to produce better assembly out? need side effects? Multiple inheritance? Want to write an OS? CL will deliver the goods.
In short, i would guess that from a computer scientist or reseacher point of view, Racket is certainly more atttactive, but for the engineer or start-up owner that wants to have killer production systems done in short time, or to create really complex, innovative systems that can be deployed to the real world, Common Lisp ought to be the weapon of choice!
What? There's five points on the numbered list, #4 is entirely about speed.
http://www.iaeng.org/IJCS/issues_v32/issue_4/IJCS_32_4_19.pd...
And now there is also CLASP (LLVM based), great for interfacing with C++ libraries. https://github.com/drmeister/clasp
It needs a review of the CL environments that survived to modern days and adopt the common extensions.
- sockets via usocket
- threading via bordeaux-threads
- metaobject protocol via closer-mop
- foreign function interface via cffi
all these are not mentioned in the standard at all, but are implemented and used in production environment.
Having base standard and extensions which may be used portably (via portability layers) is better situation than having no standard (for instance python or ruby) and a reference implementation.
Also, it means that a Lisp newbie won't be aware of what are the best libraries that are portable across Lisps for such features.
EDIT: typo correction (missing are)
> (ql:quickload 'bordeaux-threads)
> (bt:make-thread (lambda () (sleep 1) (print "hijack")))
that's all, no feature expressions whatsoever. Quicklisp is a system manager (something similar to npm in sense that it helps you to download dependencies).
Regarding best libraries: isn't that true for any language? Newbie doesn't know good libraries, because he is a newbie. He has to learn which libraries are good and which are bad.
edit: and by each supported implementation I mean in fact all active complete Common Lisp implementations: ArmedBear, Allegro, clisp, Clozure, Corman, Embeddable, Man-Kai, LispWorks, Steel-Bank and Scieener.
In any case, the Lisp like languages I use aren't Common Lisp (Clojure, Emacs Lisp, Script-Fu), so that was just an idea.
Please detail, because i'm sure that the Lisp community would love to fill those gaps. After all, Lisp is addictive to the point of resembling a hard drug.
Is it though?
I agree CL has (after slightly over 2 decades) worked out a standardization path for some core features that you simply cannot live without (e.g., usocket).
But all these variants are noticeably dated abstractions and in many cases the more modern expressions (e.g., threading vs more modern concurrency tooling) can't (or don't) use the compat libraries.
I'd rather have one good, crossplatform reference implementation than many competing but slightly incompatible implementations that only agree on very old standards. In that sense, I'd rather use Racket than CL.
It's true, letting go of some CL features is painful and you'll never get them back. CL's condition-restarts are an especially poignant loss in this modern era of Golang's almost comically regressive error handling schemes.
I'd like to think that if CL'er start to disperse more into other language communities that their gentle pressure and eloquent examples can help motivate other communities to adopt these features.
You don't need database providers to be integrated in the language spec. Not in CL and not in many other successful languages.
We already have a de facto common, portable FFI: CFFI.
We already have more than one library to do threading in a portable way. Also, threading has been left out of the CL standard on purpose, so each implementation can offer different capabilities and you can choose whatever fits best to your problem.
> The biggest issue I see with Common Lisp is that the standard is stuck in time,
The standard is from 1994 and Lisp is still pretty much one of the most modern and advanced languages around; i would say it's the other languages, like Java, that historically get new 'versions' each few years because they can't be extended using the very same language, unlike in Lisp or other modern languages like Julia or Racket.
Many people underestimate how useful Perl DBI, JDBC, ODBC, Python DB-API, ADO.NET are.
Which is why, in spite of all design flaws, even Go has a database interface defined on their core library.
> Lisp is still pretty much one of the most modern and advanced languages around
I agree with this part, and we are still far from the whole Symbolics experience.
Yet using threading as an example, since it is left for each implementation, it means one cannot guarantee portable semantics across implementations.
Exactly the issue we had with writing threads in C before p_threads came to be, and even as portable library there are semantic issues (e.g. signal handling) until C11 finally defined what threads in C are supposed to look like.
JDBC, ODBC, ADO.NET are not part of the language spec; they are separate specifications. Your initial argument was that the language spec ("the standard") is "stuck in time".
And on the other hand, database access is not really an issue on CL.
> Yet using threading as an example, since it is left for each implementation, it means one cannot guarantee portable semantics across implementations.
There does not exist a single, one-size-fits-all, valid-for-all-use-cases approach to concurrent programming.
And in CL there does not exist a common platform to efficiently build better abstractions in CL. TBF, this is only marginally worse than many other popular languages. Still, it's undeniably worse.
https://wiki.python.org/moin/PythonImplementations
There is a reference implementation, no standard, and a bunch of implementations and/or extensions. Is it really better? What guarantees do you have? If you need to provide an extension, you have to discuss about extensions and push patches to CPython, just like you could push patches to SBCL or ABCL.
But at least (which I do not consider "undeniably worse"), CL is its own platform and offers a standard extension mechanism (e.g. compiler-macros), and implementations offer more specific extensions mechanisms (sb-vm, ...), as well as a standard way of building portability layers across implementations. Those features are used to define and implement new standards. How is that "undeniably worse"?
So why are they relevant here?
Not that I like Python, mind you. I miss CL, but I wouldn't go back to it. My experience is that way too many of these beloved libraries have an audience of 1 or 2, sometimes a single company. If I'm going to engage with an environment like that, I'd rather do so to access the leading edge of research and compilation results like w/ Haskell. Even Racket is more desirable than CL on this scale.
EDIT: I just wanna repeat I loathe python and carry no water for that standing wave of bad design decisions. Really. Seriously. Worst.
lparallel and fset's affordances are not. I'm not sure why a single threaded regular expression compiler is mentioned here.
I'd have to check. Is fset even using the current standard (not the newest stuff) for immutable-friendly data structures? Last time I checked they had used a lot of older stuff from Okasaki's work and much of that has been improved upon substantially now.
Even Clojure is out of date, compared to this year's innovations!
BORDEAUX-THREADS is a proposed standard for a minimal MP/Threading interface. It is similar to the CLIM-SYS threading and lock support, but for the following broad differences:
Some behaviours are defined in additional detail: attention has been given to special variable interaction, whether and when cleanup forms are run. Some behaviours are defined in less detail: an implementation that does not support multiple threads is not required to use a new list (nil) for a lock, for example.
Many functions which would be difficult, dangerous or inefficient to provide on some implementations have been removed. Chiefly these are functions such as thread-wait which expect for efficiency that the thread scheduler is written in Lisp and 'hookable', which can't sensibly be done if the scheduler is external to the Lisp image, or the system has more than one CPU.
Unbalanced ACQUIRE-LOCK and RELEASE-LOCK functions have been added.
Posix-style condition variables have been added, as it's not otherwise possible to implement them correctly using the other operations that are specified.
Threads may be implemented using whatever applicable techniques are provided by the operating system: user-space scheduling, kernel-based LWPs or anything else that does the job.
Some parts of this specification can also be implemented in a Lisp that does not support multiple threads. Thread creation and some thread inspection operations will not work, but the locking functions are still present (though they may do nothing) so that thread-safe code can be compiled on both multithread and single-thread implementations without need of conditionals.
To avoid conflict with existing MP/threading interfaces in implementations, these symbols live in the BORDEAUX-THREADS package. Implementations and/or users may also make them visible or exported in other more traditionally named packages.")
From a practical point of view, I consider portability layers to be sufficient; but having those additional standards offer stronger guarantees across implementations.
https://trac.common-lisp.net/bordeaux-threads/wiki/ApiDocume...
https://common-lisp.net/project/cffi/spec/cffi-sys-spec.html...
- Armed Bear Common Lisp (ABCL)
- Allegro Common Lisp (ACL)
- CLISP
- Clozure CL
- CMUCL
- Corman Lisp
- Embeddable Common Lisp (ECL)
- LispWorks
- MCL
- MKCL
- Steel Bank Common Lisp (SBCL)
- Scieneer CLIt is useful, but Go is less standardized than CL. If we compare features that implementations provide, then we can cite LispWorks's Common SQL (http://www.lispworks.com/documentation/sql-tutorial/index.ht...).
if CL have a lot of libraries, it is not because CL have a large or active community, it is more because it existed for a long time
so, what is honestly the reality of CL ecosystem, is quicklisp a solid part of this ecosystem, is it easy to find and install packaged compared to perl and python for example?
do you think racket have a more reliable ecosystem?
Yes, it is very popular. You can also, without Quicklisp, download manually the project and let ASDF do the rest of steps.
> is it easy to find and install packaged compared to perl and python for example?
There is this CL environment, Portacle (The Portable CL Environment) which you just download and immediately gives you (no more steps needed):
- SBCL lisp implementation/compiler
- Quicklisp
- SLIME environment
- Customized EMACS
- and other nice tools.
Quicklisp is really easy to use, as easy as, say, PIP for Python.
However, where it fails for me is in its lack of interactive development. When I investigated it, there seemed to be no way to actually connect a repl to a running program.
Unlike with common lisp or clojure, with racket if you make changes to your code you have to restart the REPL, which destroys your state.
This was a big disappointment to me, because even python with ipython and autoreload allows for more interactive development.
I suspect that this decision was made because of racket's start as a teaching language, because it is simpler, but way less powerful.
i (poorly) implemented conditions for racket once, including the ability to drop into a repl at the point of error.
so i think you could put in place some macros that would let you get at a repl wherever you wanted. probably not the sort of thing to leave in place all the time, though
> Unlike with common lisp or clojure, with racket if you make changes to your code you have to restart the REPL, which destroys your state.
that's a different issue. i believe matthias has commented on the mailing list (some years back) that the semantics associated with doing something like that are a mess, and that's why they weren't interested in implementing it.
My workflow is generally to build up state, and then experiment with functions on that state until I get the correct output.
This workflow is very natural in Clojure, Common Lisp and even Python (with IPython and autoreload).
However, in Racket you have to restart everything on every change. This works ok for smaller applications, but if for example, your state is a large dataset that you pull from a remote database, it becomes a little more difficult.
There are possibly workarounds, and I'm not saying that Racket is bad because of this. There are definite advantages to this approach, mainly for keeping everything simple and predictable. However, this was a roadblock for me, and the main reason why I didn't spend more time working on it.
I also miss the ability to make changes to code, refresh the browser and instantly see the changes (instead of having to restart the server after each change).
Here's a good discussion (in my opinion) of what makes a good REPL for interactive development: http://vvvvalvalval.github.io/posts/what-makes-a-good-repl.h...
https://docs.racket-lang.org/drracket/Keyboard_Shortcuts.htm...
http://docs.racket-lang.org/guide/eval.html?q=namespace#%28p...
Before that I would use VIM with a tiled window manager that worked really well.
It pretty much keeps me from using Racket, and a bunch of other things that are otherwise very nice. Given a choice, I will always choose the tools that support me in building things by modifying programs as they run. I'm just happier and more productive that way.
So one of my axes of optimization is selecting projects that enable me to work that way.
I still might build a bard on Racket, but if so, I'll most likely build my own VM and implement my own image-saving solution for it.
Assuming, of course, that there are enough hours in the day, and enough years in a life.
Even the article's list of areas where Racket is "vastly superior" is questionable, IME. Granted, the author wrote ASDF, so he has a very different perspective than I do on the module system, but in practice nothing on that list has been a problem for me, and a few of them I'd actually consider to be anti-features (like a built-in GUI library).
Outside of work it's my go-to language for learning new stuff. I use it for a bunch of little programming projects that I put on Github (https://github.com/jl2?tab=repositories). They're mostly just playing around with whatever topic I'm interested in at a given time. Lots of animations and (simple) computer graphics, some simple games, one or two libraries for the Raspberry Pi, etc. Nothing too exciting.
I want to start contributing to some of the open source libraries and projects (like stumpwm and sbcl) that I use a lot, but so far the few commits I've made to other projects have been really small bug fixes or trivial documentation fixes.
Syntax case vs. syntax parse isn't and will never be close to a fair comparison. Not only is it more powerful, it also provides the users of your macros with proper error messages. It blows both unhygienic and other hygienic macro systems out of the water for anything more complex than very basic macros.
Interesting system indeed
More info here - https://www.reddit.com/r/Racket/comments/5g8xse/are_there_an...
Arc is not Racket.
Last time I checked CL images were huge though. Something like 24MB for a "hello world" executable, even bigger with some compilers.
$ cat > hello-world.lisp
(defun main () (format t "Hello, World!~%"))
$ sbcl
* (load "hello-world.lisp")
T
* (save-lisp-and-die "hello-world" :toplevel #'main :executable t)
[undoing binding stack and other enclosing state... done]
[defragmenting immobile space... 1110+19908+28500+22601 objects... done]
[saving current Lisp image into hello-world:
writing 4816 bytes from the read-only space at 0x20000000
writing 2320 bytes from the static space at 0x20100000
writing 2383872 bytes from the immobile space at 0x20300000
writing 15190112 bytes from the immobile space at 0x21b00000
writing 39911424 bytes from the dynamic space at 0x1000000000
done]
$ ./hello-world
Hello, World!
$ ls -lh hello-world
-rwxr-xr-x 1 user user 56M Aug 23 10:23 hello-worldI wrote TXR to basically be the "acceptable {awk, perl, ruby, Python, ...}" for Lisp-minded people. Well, for me, that is; but anyone else too.
With Debian it should work out of the box, on other distributions it probably needs to register a wrapper/trampoline with the binfmt_misc infrastructure.
FWIK running instances don't share the same core in RAM though.
$ cat hello-world.lisp /tmp
(print "Hello. World!")
$ sbcl --noinform /tmp
* (compile-file "hello-world.lisp" :output-file "hello-world")
; compiling file "/tmp/hello-world.lisp" (written 23 AUG 2017 11:09:24 AM):
; compiling (PRINT "Hello. World!")
; /tmp/hello-world.fasl written
; compilation finished in 0:00:00.001
#P"/tmp/hello-world.fasl"
NIL
NIL
* %
$ chmod +x ./hello-world.fasl /tmp
$ ls -lh hello-world.fasl /tmp
-rwxr-xr-x 1 user user 330 Aug 23 11:17 hello-world.fasl*
$ time ./hello-world.fasl /tmp
"Hello. World!" ./hello-world.fasl 0.00s user 0.00s system 44% cpu 0.009 total
$ /tmp
You need to consider that Common Lisp includes a runtime, and this is a essential part of the goods of Common Lisp. Whenever you create a Common Lisp program, you also "embed" the runtime, and the runtime allows you to do wonderful stuff within the running code, like for example, compile code on the fly, replace function with new version (while the program is running), replace class instances with new versions of the class (while the program is running), all sorts of enforcements of the required data type for actual data* (while the program is running),
and,
it has the wonderful "conditions/restart" system. The combination of the latter with all the former, allows you to 'correct' or 'patch' a running program without really needing to recompile the whole program or even restart the running program. Famous story of usage of this feature is using it to patch a bug on the Common Lisp program for the autopilot of the NASA Deep Space One spaceship. The NASA engineers were able to debug and then fix the program while having the DS1 in space with the code running.
So, the FASL (fast load) file is basically similar to the memory file created whenever you put a laptop into "hibernation" mode; it allows you to quickly load the whole runtime, plus your code, *plus the instances of your objects or data, if needed, just straight into memory. This is how "fast load" is achieved.
Actual RAM usage of Lisp will vary with implementations; again, as I mentioned in another post, you pick the implementation that suits your particular needs.
Lots of other people do this without massive executables to distribute, though. CLs are inconsistent on how they can distribute base images and executables. For example, Java's compiler is a code segment you an link in if you want to carry it with you.
There is only one Common Lisp, the 1994 ANSI Standard.
There are many implementations and they give you diverse options of delivering. For example if you want good options to package an executable, I think LispWorks has lots of options for specifically this need.
> For example if you want good options to package an executable, I think LispWorks has lots of features.
It had 2 alternatives. I know, because I shipped some stuff to prod for Powerset on LW back in like 2009? We ultimately stopped using it precisely because we shipped less bytes more quickly with Java and Ruby.
> As for Java, to execute a Java class you need a JRE (java runtime environment), which is not small at all, so I am not sure if it gives you a big advantage.
I'm surprised to hear you say this, but in the interests of completeness I'll give you a handy way to compute the difference in bytes saved.
(defun bytes-saved (num-deploys stdenv-size)
(* (-1 num-deploys) stdenv-size))
This requirement is even more important in the world of diverse deployment techniques that expect modularized executables as a space and performance optimization. Docker, Nim, deployments around FreeBSD jails, and even package managers have dramatically improved performance if you can offer that kind of segmentation.
http://www.oracle.com/technetwork/java/javase/windows-disksp...
ONE installation is inevitable. It's that very few CL-offering environments give you anything better than an image.
I have experience deploying with Clozure on Windows. The raw memory images are large-ish, but they compress extremely well into the installer (such as NSIS). I think there is a lot of wasted space in there. Lisp images are not unlike Unix core dumps in may respects. It's similar to checkpointing a process to disk. There is a time versus speed tradeoff there; if you compress it, then you can't just map it into memory as-is.