A Tour of the Lisps
fosskers.ca
fosskers.ca
Scheme itself has a reputation for being very "minimal", and R7RS itself is great, but it also shines as a base for bigger languages like Guile and Gauche.
Clojure is pretty cool too though. I love that it really has a life of its own as a "third" dialect to complement Common Lisp and Scheme.
Can't speak for Elisp, not enough room in my brain to learn Emacs after years of Vim.
If you like Python, I'd also like to shout out Hy (https://hylang.org/), which compiles to Python, and like Fennel is to Lua, it does a reasonably good job of bolting on s-expressions while preserving the semantics of the underlying language. It's definitely a "bigger" language than Fennel, but Python is similarly "bigger" than Lua. Hy has also developed its own distinct feel, especially when you take into account the first-party Hyrule utility library (https://hyrule.readthedocs.io/en/master/index.html) which provides a lot of very interesting macros, deriving lots of useful ideas from both Common Lisp and Clojure.
Hy is nice, but "compiling to python" isn't
I'll try and find it
It wouldn't be very difficult to provide an implementation of `let` that behaves how you would like though, so there is that at least.
I'm looking for a compiled Scheme or Lisp. By that I mean, I want "prog.ext" to create the executable "prog".
I want this because I want to make some command line utilities, and I would like the actual code compiled (vs some p-code bundled with an interpreter).
I had tried to get started with Guile, as all of its packages looked attractive, but was stymied. It's been awhile, so I can't express details. It had to do with a combination of things like packages, modules, and I know I ran into a version issue (I think I wasn't running the latest version from my package manager, so it didn't have something in the documentation).
Anyway, for such an extensive system, I found it surprisingly frustrating.
I've been trying Gambit, which seems like it would be a really nice fit, but it's fighting with my Mac, because macOS doesn't put stuff in /usr/include (wchar.h in this case), and all of my efforts to fix that have failed (save I have not tried upgrading to whatever the latest macOS is).
Maybe I can try ECL, maybe that will work.
But, anyway, Guile, while attractive, just surprised me at how I wasn't really able to make it work for me enough to make me look elsewhere.
It’s real highlights are very powerful macros and the ability to override the reader, so you can effectively create arbitrary languages (examples include reimplementations of Java, Lua, and Datalog, and a documentation generator with embedded Racket called Scribble). IMO it’s a research language first and foremost. But it has unusually good production support and online resources (I mean it when I say it’s easy to setup and learn), so I think it fits everything you asked for.
I’ve seen this comparison before, so clearly Janet isn’t doing a good job of explaining itself, but I think the only thing Janet and Scheme have in common are a few parentheses. Different core data structures, different feelings about mutability, completely different macro system…
Guile and Janet share PEGs (sorta) and embeddability but I didn’t think those were standardized at all. (I don’t really know any schemes.)
I've advertised Gauche frequently here on HN (including in another comment in this thread). I have no affiliation with the project, but I like it a lot and I think it's under-appreciated. Its author hosts its documentation under the domain name "practical-scheme.net" (https://practical-scheme.net/gauche/man/gauche-refe/index.ht...) and I think the name is very well deserved.
I remember fighting with Guile a bit to achieve the same goal, although I don't remember the details at this point, and my experience might be outdated now. I do remember that some things which are included in Gauche (e.g. JSON) are 3rd-party libraries in Guile. Also I don't understand why some of their libraries are called "ice9". What's with the Cat's Cradle reference? Is it a pun? It it someone's username? Is it a "contrib" library of some kind that was absorbed into the standard library? Why didn't the name get changed?
sbcl compiles and generates an executable quite nicely.
[0] https://www.gnu.org/prep/standards/html_node/Source-Language... [1] https://www.gnu.org/manual/blurbs.html
An interpreted lisp could fulfill this requirement. I created one such lisp. I came up with an ELF code embedding method that lets me add lisp code to the interpreter executable itself. I can add lisp modules and they become loadable. I can add a main function and it gets executed automatically.
I wrote about it here:
https://www.matheusmoreira.com/articles/self-contained-lone-...
The lisp itself is not ready to be used for anything serious at the moment. I'd really like to see this ELF embedding show up in other languages though. It's the best solution I found to the perennial self-contained executable problem.
Creating an sbcl executable for Linux is simply a call to dump an application, add the runtime and save that to some place. The there is an executable on the disk. The only thing is that the executable is not small or tiny -> but they include the compiler/loader/repl/..., which is useful.
Many other Lisps also can create executables. A LispWorks executable application on my Linux/ARM64 starts with roughly 7MB and it has unused stuff removed by a "treeshaker".
I agree with you. That's what I tried to express in my post. They asked for a compiled lisp but the actual requirement was self contained executables which interpreted languages could also fulfill.
It seems there was a period where Guile was considered a serious contender for the space now occupied by Python. My impression is that GNU lost mindshare and the opportunity because the Guile documentation takes a simple, elegant language and makes it inaccessible for newcomers. Python was known for being well-documented and easy to approach.
I'm curious what other's take on the history is.
What's wrong with p-code bundled with an interpreter?
Some libc+cc combos add some code to do things the target architecture doesn't support natively, like ints of certain sizes. From this to the example you said, where in this spectrum do you consider it not to be compiled anymore?
Unless you're debugging the emitted binary instructions, why does the compiled output matter if it reaches the desired CPU and memory requirements?
That let me write linear async-await-ish logic code in Guile and then have all the I/O and grungy stuff handled by an event driven layer in Perl space.
(this was about 20 years ago, give or take, but I still periodically get accused of writing lisp in whatever language I'm implementing things in at the time ;)
Chez is good because it's supported by Akku, but I'm not sure if it will ever support R7RS. It does have a really nice FFI though, and the docs are very good.
R5RS is too small for real work unless you add tons of SRFIs, so they created R6RS for people who want to get stuff done. But the the R5RS people got ticked off because "its too big".
R7RS was supposed to be a compromise with a tiny R7RS-small, but later adding most of the R6RS features with R7RS-large. It's now been a decade and R7RS-large seems to be completely dead.
R7RS-Large development is slow, but not at all dead.
Stackoverflow is created.
The world is running on Windows Vista (XP actually, but that's another story).
Github is created.
Facebook pulls ahead of MySpace in user count for the first time.
Blu-ray finally beats HD-DVD.
People think the Large Hadron Collider will end the world when it turns on.
Randy Pausch's book "The Last Lecture" becomes a New York Times best seller.
First Android phone and iPhone 3g release this year with ARM11 CPUs. The App Store is brand new.
JS is still slow and Chrome releases to change that.
Amazon buys Audible.
Rust essentially doesn't exist.
AirBnb is founded.
Bitcoin doesn't exist yet (though the paper drops this year).
Memristor is finally proven to be possible.
The 46th Mersenne Prime is found.
USB-3 spec drops for companies to start working on.
Core i7, and Atom on 45nm are the latest CPUs (AMD also launched Phenom on 65nm). GTX280 and HD4870x2 are the fastest GPUs around and people are discussing how overpriced top-end GPUs are at $450-550 and complaining that these GPUs use 200-250w of power.
Tesla Roadster releases for about $100,000 and Musk launches the first car (promised to the actual founder) into space out of spite.
Not everything evolves at the speed of TL;DR; attention span folks wish for.
Even Common Lisp which languished for a terribly long time due to infighting didn't take that long despite being a much more comprehensive and difficult spec.
Just as an example, C++ concepts were originally presented in 2005 [0], dropped in 2009 [1], redesigned as Concepts Lite in 2013 [2], graduated to technical specification in 2015 [3], added to C++20 roadmap in 2020 [4].
Making it 15 years to fully work out a language feature and related library functions, and even what came out 15 years is a subset of the original proposal, not going to bother with specification refinements after C++20.
Everything goes faster if it were us doing it, right?
[0] - https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n17...
[1] - https://isocpp.org/wiki/faq/cpp0x-concepts-history
[2] - https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n37...
[3] - https://en.cppreference.com/w/cpp/experimental/constraints
The interesting thing for me (a while back) was realizing that even doing something as simple as an HTTPS request would be a bit of a challenge: https://taoofmac.com/space/blog/2019/06/20/2310
For me, the underutilized gold mine feels like logic languages (Prolog et al.) Though I know recursion is used there a lot, too.
It's the same reason why I use let instead of a function when I want to encapsulate simple state across multiple statements. Sure they are both the same fundamentally, but one is much easier to reason about.
But, again, it's probably because I've been exposed to too much Scheme that this sits so prominently in my mind. I've never worked in a large scale Common Lisp code base.
Not really.
Lisp has imperative loops which process lists: prog/go, do, dolist, loop for in/on.
Then it has a bunch of higher order functions: mapcar, map, mapc, member, find, remove, delete, every, some, ...
For example this is the equivalent of an early 1960s procedure to map a function over a list for side-effects:
CL-USER 5 > (defun mapc-old (x f)
(prog ((m x))
loop
(when (null m) (return nil))
(funcall f (car m))
(pop m)
(go loop)))
MAPC-OLD
We can then process a list. Here we print the squares of the elements.
CL-USER 6 > (mapc-old '(1 2 3 4)
(lambda (e)
(print (expt e 2))))
1
4
9
16
NIL
CL-USER 12 > (dolist (e '(1 2 3 4))
(print (expt e 2)))
1
4
9
16
NIL
CL-USER 17 > (loop for e in '(1 2 3 4)
do (print (expt e 2)))
1
4
9
16
NIL
In practice, you want to recurse only on the elements of a list. And that's just how you'd handle a tree whose nodes are vectors instead of lists. So the implication that lists lead to recursion is a red herring.
Scheme is misleading because of its fetish of implementing iteration with tail calls.
https://stackoverflow.com/questions/49912204/why-there-is-no...
The SICP book had a paragraph on the distinction before recursive process and recursive procedure. I can't remember which was which, but the point there was that recursive procedure could be an iterative process if it's written with tail recursion in mind, in which case it would not overflow the stack.
I've never worked in the source tree of a large Common Lisp program. Though I've bought the books and read the tutorials etc.
Recursion appeals to me because of immutability:
I do not want to do a summation by writing the wrong answer to memory, and then updating it in-place until it's the right answer. It makes me have to think not only about what the right answer is, but when it is.
But can't I just use stack variables? Well, I don't think summation of primitive ints using stack variables is too taxing, but what about computing a word-count instead of a summation? Then your 'stack variables' are pointers into Strings and mutable HashMaps. When you return the result, do you make a defensive copy? How deeply do you copy? Your clone of a HashMap can still reference mutable data in your original HashMap.
Do you name your variable 'result' when it does not yet contain the result?
You might have to think about the primitive/object split - when is it pass-by-value and when is it pass-by-pointer? Do you have in-values and out-values? With immutability, you only have values, and you only pass-by-value.
If you're sharing objects across threads, and you decide to lock - is that sufficient? What if the caller takes the lock, does a get(), releases the lock, and then starts mutating the gotten value, bypassing the protection of the lock. Now you're thinking about defensive copies and maybe deadlocks too. Just share an immutable value without a lock.
Sometimes recursion is aesthetically better, but usually it's just what you're left with after you take a big hammer to mutability and its surrounding issues.
And most of the time (in Haskell but I'm guessing in Lisps too) you don't write out direct recursions, but instead use maps and folds which have recursion under the hood.
A function should be a "black box". I don't care what happens inside it as long its externally observable behavior is pure.
On "classical" computing hardware, it's fine. On present-day architectures we're better off writing with "wider" operations across vectors rather than depth-first-ish recursive operations on scalars.
But yes, for some domains, the performance doesn't matter.
I just also find recursion more difficult to read in some contexts. In others, things like tree transformation etc. it's easier to read, for sure.
But I am also a fan of immutable, or at least copy-on-write, data, expressed in series of functional applications, etc. etc.
The Reasoned Schemer is an accessible introduction to that space if you’re not put off by the socratic-themed dialogue.
& the Barliman demo is still pretty exciting even after LLM codegen.
If you want to write one yourself, it's pretty easy: https://www.youtube.com/watch?v=y1bVJOAfhKY
Scheme likes to tout it because of tail call optimization. Common Lisp can do it, but doesn't really shove it as a first principle of "Common Lisping" (and CL does not dictate tail call optimization).
I don't use it (save when it's necessary). I just iterate using the supplied iterators (do, dolist, loop, whatever).
I can't speak to other functional languages.
Recursion is sometimes the easiest way to reason about a problem though, so it's nice to have decent support and TCO.
Yes, it's one of the things that allow FP algorithms to be more declarative. It's difficult to seriously argue that recursion is fundamentally more complex-- the contrary argument is more obvious to me.
Everything that's left is the fact that it requires people to unlearn things, which is uncomfortable and requires effort, but unfamiliarity is not an indicator of complexity.
I still "see the parentheses", and I wish there was a way (outside Coalton) to integrate the security that a strong typechecker can provide, but I really do enjoy working in the Lisp ecosystems now, and feel like I understand Lispers excitement at what they have.
Have you checked out typed-racket? That's a reasonably nice way to get strong typing by tackling type correctness contracts on top of otherwise standard racket.
There's also Alexis King's marvellous Hackett, which seems to be finished for now, but was positive proof that you can have - and eat - all the curry-flavoured cake you want while staying in scheme.
The parentheses are a serialisation format; there are many alternatives using indentation, braces, infix application, etc. (e.g. those listed at http://www.chriswarbo.net/blog/2017-08-29-s_expressions.html ). The nice thing is, s-expressions are so simple that we can usually convert between them automatically (e.g. piping our indentation-based code through an indentation-to-parentheses converter before it hits an interpreter).
However, despite having all these alternative syntaxen, the fact that the vast majority of Lisp code is still written with parenthesised s-expressions tells me that it's really a non-issue :)
Common lisp is optionally strongly typed. SBCL has extensive type checking facilities. Coalton is statically typed.
Security does not imply strong (or static) typing, and strong typing definitely does not imply security (thinking your code is secure just because you used strong or static typing is silly). In fact code complexity that static typing introduces can have detrimental effects to overall security.
Helpful starting point on state of type checking in common lisp: https://lispcookbook.github.io/cl-cookbook/type.html
enhancement: functions with declared return types have their return values type-checked in optimization regimes with high SAFETY and (DEBUG 3).
Being able to inter-opt with java is great, as I can wrap scheme procedures in functional interfaces and use them as drop in replacements for java functions, as my event system was already based on predicates and consumers for event handling.
I've worked through some of the SICP and have always wanted to get more into scheme/lisp, but the barrier of starting a full project in it always kept me from getting much hands on experience. Its been quite enlightening actually getting to work with a form of REPL driven development and getting my hands dirty with coding some scheme, having access to the JVM means I can do practically anything with it, with out needing to bootstrap tons of code, and using it in a project with a large scope lets me solve real-world problems with it vs just toying around which has been what most of my scheme/lisp experience was before.
After Kawa, I'd pick ABCL. It's not as fast as Kawa and its interoperability isn't nearly as good, but it is effectively a 100% standards-compliant Common Lisp: what more could you ask for?
Only last would I pick Clojure. It's designed to feel "sort of immutable" but this is impossible if you want any degree of interoperability with the JVM. As a result, you wind up with lots of Refs and other fun stuff which in my experience make Clojure much slower than Kawa and ABCL. It's always slower by quite a lot, but in fact I've had a few extreme situations where Clojure would wind up being __literally__ three orders of magnitude slower.
But I never got much into REPL driven development. Probably because I just couldn't wrap my head around emacs.
¯\_(ツ)_/¯
One change that I made in ObjS is that I added the pipe so you don't need to parenthesize chained keyword messages.
((receiver msg1:arg1) chained1:arg2) chained2:arg3
receiver msg1:arg1 | chained1:arg2 | chained2:arg3
I can probably pull all that off syntactically, not sure about aesthetically.
1. no more keyword parameters
2. every message has exactly one argument, which may be a tuple.
3. "obj.msg" becomes short hand for "obj msg ()"
4. if a message expects 2 element tuple and you give it a one element one.. it returns a closure that expects the rest of the arguments. ie "add1 = 1.+"
"I think to try to specify all that a program should do, you get specifications that are themselves so complicated that you're not longer confident that they say what you intended."
"dynamic languages are still interesting and important. There are programs you can write which can't be typed by a particular type system but which nevertheless don't "go wrong" at runtime, which is the gold standard - don't segfault, don't add integers to characters. They're just fine."
Today I work with python (sigh) and every day I long for Clojure.
But Algols, including Python, are "bureaucrat" languages: rather than condensing the syntax to be the exact specification of the program, they define a right way to form the expression, and then the little man inside the compiler rubber stamps it when you press the run button. In other words, they favor defining a semantics and then adding a syntax around that, which means that they need more syntax, it's harder to explain the behavior of a syntactical construction, and it's harder to extend to have new syntax. But by being consistent in a certain form-filling way, they enable a team to collaborate and hand off relatively more code.
IMHO, a perfectly reasonable approach I'm exploring now for personal work is to have a Lisp(or something that comes close enough in size, dynamic behavior, and convenience, like Lua) targeting a Forth. The Forth is there to be a stack machine with a REPL. You can extend the Forth upwards a little bit because it can metaprogram, or downwards to access the machine. It is better for development than a generic bytecode VM because it ships in a bootstrappable form, with everything you need for debugging and extension - it is there to be the layer that talks to the machine, as directly as possible, so nothing is hidden behind a specialized protocol. And you can use the Lisp to be the compiler, to add the rubber-stamping semantics, work through resource tracking issues, do garbage collection and complicated string parsing, and generate "dumb" Forth code where it's called for. That creates a nice mixture of legibility and configurability, where you can address the software in a nuts-and-bolts way or with "I want an algorithm generating this functionality".
I've had this idea for a while now but never got around to actually executing it. I'd love to follow your progress if you're doing it publicly.
If you look at the arguments in favor of Lisp, they'll often boil down to it being "beautiful", "elegant", "flexible", or even "magical". It's a very minimal language which allows you to do absolutely anything - a lot of which would be an absolute nightmare in most other languages. You could implement just about any programming paradigm in Lisp if you want to. I believe this makes it very appealing to computer scientists, or other people with a more mathematical background. However, this flexibility is also a massive footgun: if you're not careful your junior developer might end up reinventing the wheel a dozen times, and writing completely unmaintainable code in the process.
On the other hand, most other programming languages look kind-of the same. They are all quite opinionated about how stuff is supposed to work, with a lot of hardware details leaking into the language. However, they are very easy to learn: most of it is just "do a bunch of operations in succession" taken to the extreme. Anyone who knows C# will be able to pick up a basic understanding of C, Python, or JavaScript well within a day, and a lot of people new to programming will be able to write not-completely-terrible code within a month or two when given the right guidance. They don't need to know about all the abstractions and technical details to be a functioning member of your team.
When you're running a business, you don't care about any of that beauty or flexibility. You want code which is quick and easy to write, trivial to read, and understandable by even the worst programmer in your company. In practice that means in your comparison you'll be choosing Python over Lisp. Heck, Go was developed entirely around this principle, cutting out as many language features as possible. And because all the other companies are making the same choice there will also be way more libraries for Python, making the gap even larger.
So yeah, in a stroke of irony Lisp isn't more popular because it is better.
Nonsense. Quoth Wikipedia, "Lisp pioneered many ideas in computer science, including tree data structures, automatic storage management, dynamic typing, conditionals, higher-order functions, recursion, the self-hosting compiler, and the read–eval–print loop." These are all solutions to real world problems.
So many languages borrow features that were originally developed in Lisp.
> However, this flexibility is also a massive footgun: if you're not careful your junior developer might end up reinventing the wheel a dozen times, and writing completely unmaintainable code in the process.
Plenty of code written in Lisp looks just like your Python, Ruby, JavaScript, Java, et al programs where you're defining structures or classes, writing and calling functions, importing useful libraries, etc. Plenty of this Lisp code is just as maintainable as the non-S-expression code.
> Anyone who knows C# will be able to pick up a basic understanding of C, Python, or JavaScript well within a day
Same with Lisp. It's just:
(function arg1 arg2)
Instead of
function(arg1, arg2)
> You want code which is quick and easy to write, trivial to read, and understandable by even the worst programmer in your company.
There is plenty of "lowest common denominator" code like this written in Lisp. Much Lisp code is not buried under inscrutable macros, just like not all Java code is buried under layers of inscrutable classes.
That isn't the problem of learning Lisp. The problem with learning Lisp is
(`a ,b c)
> There is plenty of "lowest common denominator" code like this written in Lisp. Much Lisp code is not buried under inscrutable macros, just like not all Java code is buried under layers of inscrutable classes.
But the problem remains: it only takes one wizard to make reading code impossible by outsiders.
The canonical way to write a Lisp system is to define a DSL (or, should I say, system-specific language) and implement the system in that. But this means no-one outside of the language/system developers know the language, this means Lisp tends to be write-only by design - not in the line-noise meaning, but in the obscure foreign language meaning.
You certainly can not do that, but if you choose to not do that, why pick Lisp? You can argue that all software systems develop their internal system-specific languages, and I agree - but the overall strict language provides a rigid framework for anchoring understanding, something you have to conciously work for with Lisp. (Note Java pre-2020 was so limited in expressivity it wasn't any better, because all the complexity ended up in the class hierarchies you reference - it was more similar to Lisp in that way than many realize.)
One Java architect can make the code unreadable, even for himself.
> canonical way to write a Lisp system is to define a DSL
It's popular, but not canonical. But even those DSL language patterns can be learned. In many non-Lisp projects, the DSL gets implemented in C/C++.
I agree, even in my original post.
> But even those DSL language patterns can be learned. In many non-Lisp projects, the DSL gets implemented in C/C++.
The friction is everything, otherwise we'd all be programming Turing machines. In Lisp, it is zero, by design; it can be good or bad, as most things in engineering. Consequences of uncontrolled growth of a DSL vs imposing artificial process limitations for the future of the software project must be considered explicitly; this is automatic when the cost of starting a DSL is non-zero.
That's just a subtle lie popularized by Paul Graham in "Beating The Averages".
If you can understand "String ${interpolation}", you can understand list quasiquoting.
> But the problem remains: it only takes one wizard to make reading code impossible by outsiders.
This really is a Lisp meme. There are plenty of Lisp wizards like Guy Steele, Rich Hickey, and Matthew Flatt. The wizards perform the magical act of making code legible and intelligible. I have stumbled around several Clojure and Racket code bases and never felt like "I should understand this code but the features of Lisp make it impossible to know for sure." "Infinite power" macros and whatever are really only used sparingly and generally when it's impossible to achieve a goal otherwise. No one is doing (define + -).
> But this means no-one outside of the language/system developers know the language, this means Lisp tends to be write-only by design - not in the line-noise meaning, but in the obscure foreign language meaning.
I, as a Racket novice, have been able to add candlesticks [1] to the plot library without learning much about it. I have also debugged DrRacket (an IDE) to uncover that Racket GUI operations performed significantly worse if non-integer scaling was used [2]. At no point when I was going through Racket internal code did I ever feel it was write-only. In fact, it was quite convenient to modify Racket internal source code, rebuild, and test changes in a way that would be much more difficult in Java or C++.
> You certainly can not do that, but if you choose to not do that, why pick Lisp?
Built in rationals.
The ergonomics of defining [XML / JSON / etc] data as S-expressions and doing things like pattern matching on that data.
Great, coherent integration between GUIs, plots, statistics functions, and all the other bits of Racket's batteries inclusions.
You still have access to all the other great features that other languages have borrowed from Lisp like REPL development, package managers (edit: maybe package managers were not a Lisp invention), good IDE tools, etc.
It is nice to learn the meta-syntax of parentheses once and know that the code will always look like that. No need to consider if some feature is implemented as a syntactically different new keyword, annotation, function call, or whatever. It'll always be a (feature).
> something you have to conciously work for with Lisp.
Plenty of languages have style guides, linters, static analysis tools, etc. to make sure the code conforms to certain restrictions. Lisp feels no different in this regard.
[1] https://docs.racket-lang.org/plot/renderer2d.html#%28def._%2...
[2] https://github.com/racket/gui/commit/20e589c091998b0121505e2...
This is exactly right, I don't know why so many people still fail to grasp this concept.
1. They want their names to be widely recognized, so they find a popular bandwagon to hop onto.
2. Raw numbers? More people using Python means more people trying to make libraries for Python, means more libraries remaining in the race after you eliminate the crap from people who don't know how to write libraries.
Note that Python is, by now, an old language. It wasn't instantly popular, and you wouldn't have predicted it. In, say, 1999, you had to be some GNU/Linux person to even know what Python is. It was far from obvious that, of all things, it would get so popular. That Eric Raymond article in the Linux Journal around that time probably gave it a bit of a boost.
Python definitely rode on the coattails of increasing GNU/Linux popularity, too. More people using Linux started asking questions how to script this and that, and going "gack!" at shell or perl programming. It seems Python might appeal to survivors of VisualBasic shifting gears into GNU/Linux stuff.
Language popularity has little to do with how well languages are designed or how simple they actually are. Most popular languages are popular for historical reasons.
My brief text answer (focused on Clojure):
-python has been around about 20 more years than Clojure
-the advantages of Clojure vs python/etc probably aren’t nearly as big as python vs a non memory managed, compiled, static language like C. That whole generation of “scripting” languages had the wind at their backs in a way Clojure and its contemporaries never quite will (though Clojure et al tend to be much better at concurrency and parallelism and this will help a lot)
-unfamiliar syntax (not Algol like) and paradigm (not oop)- and the truth is many programmers back away slowly when a thing is too alien
-hasn’t found a niche as big as data science or scientific computing or CRUD website building - I think python has aggregated some great academic niches, at least one of which (ML/data science) exploded in popularity. Ruby had the rails community. Clojure seems to have some popularity in fintech but has no big single niche yet that it dominates.
Because libraries already exist for those languages, as well as support, vendors, and a big ecosystem, familiar syntax, and jobs. So they write libraries for languages that are already popular.
If you meant, "but why wasn't some Lisp the one that gain popularity back in the day, when C, C++, Python, and Java didn't exist or where still fresh?"
I think because:
1) it was too advanced for the procedural mindset at the time,
2) it was not sufficiently efficient in those primitive 16 bit machines
3) fragmentation
and most importantly, no killer app and major vendor backing or OS first-class support (like C had for UNIX, C++ for Windows, and Java got from SUN).
As for why things are and aren't popular, who knows? It is quite possible that people just don't like the look of the parenthesis, or there are a couple of key libraries that aren't good but nobody vocal has put there finger on which ones. The error messaging isn't quite up to a standard that people want to deal with. Or maybe popularity is really just about pure random chance, at most 4 languages get to have >20% market share, by definition.
Actually, I will make that claim without hesitation.
Programmers who program in some language for a few years realize they are still learning more about it, all the time. Therefore they know if they switched to another language it would also take them a few years to attain the same level of mastery.
And then there is similar inertia with an organization that has lots of code in some specific language already.
For some people, community signals are very important. Massive conferences or raw number of libraries, etc., indicate some inner quality of that language's ecosystem that they value.
I want to move on from Python, but there are so many more people that I can help if I stay.
I prefer Haskell's type system, but Clojure has better syntax IMO.
Imagine the best and worst DSLs that you've had to use. Joining a Lisp team would be somewhere on that spectrum though I hope their homegrown/app language is far better than the average/bad DSL.
Clojure is much better in that it has many 'batteries included' and opinions on things to make different codebases less different than with other Lisps.
Does anyone have any tips on where to find a lisp job?
[1] https://www.reddit.com/r/ProgrammingLanguages/s/AZmouaoARl
[2] https://jobs.lever.co/dodmg/af802f7f-4e44-4457-9e49-14bc47bd...
> It would be something like a fusion of Clojure and Common Lisp, (...)
Mine would be, too. Unfortunately, that's not possible. Clojure is one of those few languages actually designed by somebody who knows what they are doing. Clojure's limitations come from the limitations of the platform it is running on and certain design criteria (interoperability) that were necessary for its success.
Not saying you're wrong. But look at the authors of Scheme and the CL spec... Rich may have seen far, but he was standing on the shoulders of giants.
In fact someone seems to have started on one: https://github.com/ruricolist/cloture
If you are doing long running server side apps, it is a better fit. Even better if you are already a Java shop.
Otherwise, its either detrimental or, at the very least, a source of very 'alien' behavior, not the least of which being the stack traces. That gets pretty obvious when you compare with the likes of Common Lisp with its incredibly elegant system that's essentially Lisp _almost_ all the way down.
The JVM has its own advantages of course. Billions of dollars of optimization work being one of them. Being able to use java libraries to fill the gaps (at the expense of the less elegant stack traces and some
It can be a complete show stopper in many applications. Say, you want to interface with C libraries. Or embed some form of Lisp in your app. Browser-based apps (emscripten doesn't help you), which is why Clojurescript exists.
Or you are building something like an iOS application. I have successfully embedded (although not shipped to the App store) Chicken Scheme in a couple of different ways. The first, as a library with all the cross compilation nonsense. And the second, by simply telling the compiler to stop at C code generation, adding the C blob in the app, and compiling everything together with the rest of the app. That gave me a remote REPL which was amazing for debugging.
It isn't much fun and there are certainly situations where I'd say Clojure was a poor choice for calling C/C++ libraries, but if you need to do it then it can be done.
What do I care if it's in the JVM? Sure, a JVM instance uses a lot of memory to help the garbage collector, but that doesn't bother me. JVM is just an old, mature ecosystem. Every runtime we work with (browser, nodejs, CPython, your decades of hand-written C++, the Go standard library) shares design tradeoffs with the JVM. Nothing inherently off-putting about it.
First it’s viscerally slow. They have a state of the art GC, amazing benchmarks, tons of work going into performance, but it still feels slow and laggy when you develop on it. None of the other ecosystems you mentioned have that problem (including Python).
Second, they have a bad sense of design. The class library comes from a culture of needing three classes to open a file, and that culture permeated through the entire ecosystem. Almost all the software in it feels bloated and over engineered. The modal JVM experience is spending 95% of your time dealing with “enterprise-y” boilerplate that turns out to have nothing to do with the enterprise and everything to do with bad design decisions and the culture downstream from those. C++ has its own flavor of this problem, but certainly not Python or Go.
Although the JVM is one of the sleekest environments around though and I'm confused by the fellow saying it is "viscerally slow". Clojure loads slowly, but after that everything happens at speed.
I really can't relate to this. What part of the process is noticeably slower than Python? I have a lot of Python projects and couldn't say any of them are slower than JVM apps.
For me, it made me not miss tail-call elimination at all.
(define (foo)
(bar))
Looping construct in most languages provides a simple conversion of self recursion (a function calls itself recursively) to a loop: update the loop variables and then do the next loop iteration.
But the general case of tail calls is not covered by a simple local loop, like what is provided by Clojure.
Aside from that, I agree. Tail-call optimization's benefits are wildly overblown.
The optimization of stack frame elision is pretty large for function calls on the JVM and the stack limits are not very amenable to ‘typical’ higher order function ‘functional programming’ style.
This is more general than what tail-call-optimization can handle. This is true, but only in the context of recursive functions, and you don't actually save anything besides not needing to re-allocate the stackframes below your recursion point. Other optimizations such as inlining may perform some of this in the general case. Regardless, you get the same benefits by using `recur` in Clojure, it's just explicit, it still uses no extra stack space.
The downside is purely stylistic. It's functionally the same as if you did `(let recur () ...)` in Scheme.
I do not know if the above is included in what clojure does for tail calls, recursive or not, but on the JVM the elimination of those calls can and does have an impact.
I believe they're related, but not the same thing.
> I do not know if the above is included in what clojure does for tail calls, recursive or not, but on the JVM the elimination of those calls can and does have an impact.
As far as I know the JVM doesn't allow a program to arbitrarily modify the stack, so any support would need to be baked into the JVM itself, which it might be now, but I'm not finding any indication that it is. The `loop`/`recur` construct essentially compiles to a loop in the Java sense (to my understanding), so it is as efficient as a recursive loop with TCO. The more general tail-call elimination likely isn't possible on the JVM, but you're correct that it would likely result in a speed up.
All of this is sort of besides the point: I don't think there's much in terms of higher-order functions (which is an extremely broad category of things) that you can't do in Clojure just because it lacks TCO. At least no one has been able to give me an example or even address the point directly. Speed is not really what I'm referring to.
s -> (a,s)
Yet it’s always noted as code smell associated with “inexperienced candidates” in interviews.
For that matter, first and last too: https://medium.com/hackernoon/faster-clojure-reduce-57a10444...
The amount of paired programmers suggesting changing nths to firsts and lasts is demoralizing.
Plus lambda's will stay alive for quite a while if they are in use. So startup time is felt once, but then would be identical to any other language or runtime.
Yes, that's because print-effing one's way to understanding is also among the most crude methods of debugging. An excellent alternative to doing that is to use an actual debugger. That will also allow you to be "inside your program".
I've yet to see a C debugger which does more than a fancy version of printf. Imagine being able to compile functions on the go while still running your main program with all the state saved.
I would imagine you'd need the same thing for accessing the runtime of any production deployed application(s) in such an unfettered manner. Erlang, Lisp, C, or other?
Implementations are a bit DIY.
An interesting play would be to implement rsr7 scheme in kernel, semi-metacircular fashion.
The abstraction leaks as well. `(reduce and (list #t #t #f))` is probably an error. Calling a macro and calling a function looks the same and sometimes is the same and sometimes falls over.
Macros are sacrosanct to lisp. Hygienic macros are a crown jewel of scheme. They're still the wrong thing though.
In the beginning there was dynamic scope, possibly by accident. There were also fexpr - pass the unevaluated data and the environment, instead of passing evaluated data, which were also dynamically scoped. Sometimes called nlambda. That was difficult to program with and very difficult to compile.
Macros are easier to use than dynamically scoped fexpr. They're much easier to compile and lisp was getting a bit of a kicking for being too slow. There were some papers written, macros were the better choice, and here we are.
Lexical scoping turned up a little while after macros won the fight. Lexically scoped fexpr with first class environments are the right thing. Simpler and more capable than macros. First class environments being another thing sacrificed on the alter of performance decades ago.
Shutt noticed this and wrote about it at length. It's slightly subtle that a non-hygienic macro is a subset of fexpr. Force inline it at the caller, refuse to run various calls (e.g. pass it to functions), wrap the return value in an eval in the caller environment. A hygienic macro has to do reflective symbol renaming stuff which is also expressible as a fexpr by implementing the symbol renaming rules of scheme, which thus far I don't have the patience for.
Qualified as minor, in that the wrong thing we have is still very useful.
Disaster in the sense that we could have fexpr if history had happened in a slightly different sequence.
Opportunity in that lisp can be better. First class environments and first class macros can be done. The performance constraints of the past have been removed by hardware progress and to a lesser extent by progress in compiler design.
Neither of these ideas rings true.
While there are things that fexprs can do that macros cannot, the reverse is also true.
Macros have the opportunity to work with the source code before it is executed. Macros can issue diagnostics. Also, macros have access to the compilation environment, which can be an entirely separate machine from the run-time environment. Macros can bring in materials from auxiliary files, which would not be found on the target.
A system build on fexprs rather than macros should incorporate the concept of static checking functions. For each fexpr, two additional functions should be required:
- a function which knows how to statically code-walk the form which is the target of the fexpr. Perhaps of this form:
(lambda (form recurse-fn)
;; function identifies, inside form, all expressions that are
;; forms and calls (recurse-fn subform) for each such subform
)
Unhygienic macros do not have complicated semantics. However, it should be acknowledged that hygiene is more naturally achieved in fexprs without effort: the fexpr's own local variables and functions are very clearly separated from evaluations of argument material, that being done by an explicit eval call, using the correct environment passed into the fexpr. A macro's local variables are likewise clearly separated, but those are expansion time. A macro has to inject new variables into the generated code which a fexpr doesn't have to do: a fexpr can use its own local variables to hold run-time temporaries related to the calculation, whereas a macro cannot do that: for run-time temporaries, it has to generate code, combining that with the argument material. Those temporaries are then in the same scope, and have to be gensyms.
Common lisp's read macros are indeed a different beast. Mutating the bytes pulled from file before parsing them is more powerful than operating on the lists of symbols.
Macros imply a phase separation which fexpr do not. The compile time diagnostics and similar are an artifact of that phase separation, not of evaluation sequencing.
If one wants to build in that phase separation, I think it follows that unhygienic macros are the simple option. Hygienic macros still carry much implementation complexity.
If one wants hygienic behaviour, fexpr do that exactly as easily as lexically scoped lambdas, because they literally are lambda which pass the argument forms unevaluated. I implemented that as a bool tag on the function affecting calling convention - do you map eval across the arguments or not during the function call - and that's sufficient.
If the desired behaviour is variable capture from the calling environment, it takes more effort from a fexpr than from a macro. You write (eval env 'foo) instead of ,foo or similar.
I don't see a particular need for a code walker - in the simple case the argument gets passed around a little then evaluated, in the complicated case you need to walk the arguments the same way a macro would want to.
The phase separation thing I had forgotten about. As in warning messages during compilation so that you have invariants at runtime. Macros fit that model better. Fexpr and compilation are an uneasy combination.
(with-slots (a b c) obj
;; a b c refer to slots (assignable!)
)
I don't see how the fexpr can do that without performing the same replacement.
a, b and c are not lexical variables you can just bind and then evaluate the body. They are aliases: assigning to a sets slot a of the object.
One way is for the fexpr to contain its own eval implementation which takes an extra macro environment, indicating that when a, b and c are seen, alternative forms are to be evaluated in their place.
That just adds up to an inefficient implementation of symbol macros.
Also, how are places implemented. In particular, user-defined places. How can we have a set of fexprs like setf, push, incf, ... which can work with with arbitrary syntax denoting a place, including application-defined places.
Re your two points:
You could have "symbol fexprs", analogous to symbol macros, I guess.
For places I think the first-class solution as employed by T and others is better, and would work fine with fexprs: (set (name-of person-1) "sam") simply stands for ((setter name-of) person-1 "sam").
IOW, name-of is expected to be a reader function. Every reader function has a writer function attached to it, that we extract with (setter name-of). Then we call that writer function with the rest of the original arguments.
Speculate that such a thing did exist. Call it symbol-lambda for less confusion with the symbol-function that does exist in common lisp. It's a function that takes no arguments, being a distinct premise from a function that takes the empty list. `(apply func some-list)` would be an error. But one wishes to call it implicitly when it shows up in lists. Related to and distinct from thunks.
Thus it's a function executed by eval, not by apply. Specifically if eval calls into eval-symbol when called on symbols, one way to go is perform the environment lookup etc as usual, and then afterwards check whether the thing one found in the environment is a symbol-lambda. Something like:
(define eval-symbol-lambda
(lambda (evalled-symbol env)
(if (symbol-lambda? evalled-symbol)
(eval (open-symbol-lambda evalled-symbol) env)
evalled-symbol)))
(define eval-symbol
(lambda (symbol env)
(eval-symbol-lambda (lookup env symbol 'unknown) env)))
The fexpr equivalent could be identical (since it's mostly distinguished from lambda by not evaluating arguments and there aren't any), or maybe the fexpr form gets passed the environment and the lambda form gets passed the empty environment instead on efficiency grounds.
With-slots then turns into something like:
(let (#gensym obj)
(symbol-lambda-let
(a (slot-value #gensym 'a))
(b (slot-value #gensym 'b))
(c (slot-value #gensym 'c))
...))
However passing one of these things directly to eval, as opposed to passing a symbol bound to one to eval, is also possible. So eval picks up a third case for eval-on-function, where fexpr and lambda get returned unchanged, but symbol-lambda/fexpr/name-tbd get evaluated.
Also the evaluation of the symbol-lambda might need to know what name it was invoked through. That usually isn't the case and interacts poorly with renaming variables. That also distinguishes passing it to eval as a function from calling it by writing down a symbol that resolves to one.
Overall this leaves me with a sense that I don't quite have the proper decomposition worked through yet.
I got to something resembling this part:
> With-slots then turns into something like:
That's when I hit the problem that "turns into" is macro code generation!
If the fexpr has to do code generation on the fly and eval it, it's doing a macro's job, badly, since the macro can do it upfront.
(arguably a macro is really just a limited fexpr that's executed and the results inlined at compile time, though I mean that as 'morally equivalent' rather than making a technical claim since I haven't attempted to implement that in my fexpr based interpreters yet)
You can get somewhat close to fexpr with lazy evaluation but it turns out that's not the same thing either. Taking the arguments apart and building different things out of them is a superset of stashing it somewhere to evaluate later.
(lambda (x) ...) ; evaluate the argument in the caller, bind it to the name x
(fexpr (x) ...) ; bind the argument to the name x
(normal (x) ...) ; make a thunk out of the argument and the environment
(macro (x) ...) ; bind the argument to x then do the force inline & extra eval on call
It's interesting that macros are the one with a bunch of extra work to do after the call whereas fexpr does the bare minimum of adding stuff to the lexical environment. Lambda maps eval across the arguments, normal maps make-thunk across the arguments.
A great foot in the door is when someone is using and likes some software that you have written. If they know it came from you, and can easily find out you're available when they're ready to hire.
Of course, that's not an option for everyone -- some people simply haven't yet had the free time or paid opportunity to make open source -- and we should try not penalize people for that. But it's much better positive signal than a job-seeker approach of "I spent person-months memorizing Leetcode medium-difficulty answers, and rehearsing whiteboard interview stage presence like a rockstar."
Beware that good Lisp-family jobs are rare, and don't tend to be highest-paying. Outside of relative pay within categories like enterprise Java-shop programmer.
(Aside: From a hiring perspective, using some beloved niche language with few jobs available is also a great way to pick up mythical "10x" hires, and retain them for a long time. :) But, more seriously, it's ethical to make sure that prospective hires for unpopular niche keywords realize what they're getting into, in the current software job environment that often assumes fad-following & job-hopping strategy, and is skeptical/derisive of people whose resumes don't look like that.)
This chart I'm looking at might be broken, because mouseover is showing median salary of exactly $96,381 for all of Scala, Elixir, Clojure, "Lisp", and F#. (But somehow OCaml and Haskell didn't get jumbled in with those.)
BTW, I'm keeping in mind that people into the fringe power-user tech might be more capable than your average bear. For example, your typical person who, somehow, got years of experience with CL, IME, is a lot more capable overall than your typical person with the same number of years using Python. So, someone taking home $150K doing a Lisp at a company that lets them might have comparable skills to someone making $500K at a FAANG. (Excluding the blip when Google acquired ITA Software, which I guess brought a bunch of CL people there.)
[1] https://www.reddit.com/r/ProgrammingLanguages/s/AZmouaoARl
[2] https://jobs.lever.co/dodmg/af802f7f-4e44-4457-9e49-14bc47bd...
I should have taken strategy notes after talking to a guy at my last job who got management buy-in to rewrite a lot of Java code (for android) to Kotlin and have all new code for android be in Kotlin (before that was considered the sensible default). I think that's in general a better approach for a lot of would-be paid lispers: don't wait for or look for the lisp job, make the lisp job. Whether that's doing work where the customer doesn't care what language the thing is made in, or introducing it (some have even snuck it in -- the original clojure.jar got a lot of early success that way) to an existing work place. What I somewhat remember from my conversation was that if you can make a good technical case and have at least one other person supporting you (ideally your entire dev team as was his case), it's a lot easier to sell. No one raised bogus concerns about increasing the hiring difficulty or effort learning the new system. (I say bogus because engineers are learning all the time, and huge swathes of the industry have already had to do things like migrate from ObjC to Swift, or the various versions of JavaScript and later TypeScript + all the framework churn, switching IDEs; learning and change are quite common and a non-issue.) From other Lisp company reports, getting a new hire up to speed to be productive with the team using Common Lisp is a matter of a week or two, a small portion of the overall onboarding time a lot of new jobs have. Mastery takes longer, of course, but that's different.
If I had stayed longer at my last job I would have continued to flesh out a better demo for interactive selenium webdriver tests for our main Java application after injecting ABCL into it, it seemed like the easiest vector to get more interest from my team and other teams. It kind of sucks when you're debugging a broken test and finally hit an exception but now you have to start over again (especially if you stepped too far in the debugger), especially with heavy webdriver tests that can take a long time. The Lisp debugging experience is so much better... And when writing the test from scratch, it's very interactive, you type code and execute it and verify the browser did what you intended. When you're done you run it again from scratch to verify.
I have been happily using Lisp languages since the 1970s and CL since 1982. I retired several months ago but I still spend time coding for my own research projects and for my book examples. I have been torn between going all in using just one of Racket or Common Lisp. Racket is a fun environment and I like the community but I have so much history with CL. I don’t know if I will choose just one, but it would be a good idea!
The article didn’t mention the Hy language that compiles to Python AST. For a while I was very enthusiastic about Hy and I wrote a book about it. It used to have a “let” macro that mapped Lisp variable scoping to the jankyness of Python but that support has gone away and I feel like using “setv” and just accepting Python like scoping is a bridge too far for me. The philosophy behind Hy is great, but it is not my philosophy.
I love the Clojure language (and I donated money to the project at the beginning) but I only use Clojure when I am being paid to use it for specific projects. The Bork Dude’s Babashka language is a fast loading Clojure, and I had a lot of fun for a while setting up a nice dev environment and using it for all recreational programming for a week. Babashka is worth a serious look.
Which won me over originally was the documentation, and the C functions following the same format as my own. Being able to add Guile into my game did really speed up my time "hacking" 3D scenes and features, before converting them over to C code.
Then started to use GNU Guile for building a personal website, and other toy projects on my server.
While I don't use GNU Guile for much these days, I certainly think it should be given, atleast, half the reputation Python gets today.
I think the problem with GNU Guile is that (last time I checked) was that it was very Linux focused. Always had trouble with it on Windows. I dont own a Mac but assume to be the same there as well.
Of course, I have GNU Guile on my Windows machine thanks to WSL2.
Like the other commenter I completely agree that knowing Haskell makes you a better programmer, and when I code in other languages I really miss certain parts of it. On the other hand, the entire Haskell ecosystem feels like a half-finished PhD thesis, and often-promoted libraries suffer from fundamental flaws which are "open research questions". There's a constant drive in many Haskellers to make their code as abstract, type-safe, and generic as possible, but unfortunately practicality and ease of use are often forgotten along the way.
It's a great language, but if you're trying to write production code which builds upon the wider ecosystem you're in for a world of pain. In 2024 Rust has probably rifled enough through Haskell's pockets to make it mostly irrelevant as anything but a testbed for programming language researchers.
(The actual reason I left Haskell was because I switched my employer. Choosing an employer is IMO a bit more important than choosing the language to code in.)
The "functional core, imperative shell" approach is -fantastic- though and I'm not sure where I picked it up from but a lot of my (especially async/event driven) code looks at least close to that.
It occurs to me that this may be why I've yet to be significantly bothered by function colouring when writing async/await based code in Perl and/or JS, but I'd probably have to give that more thought before I go beyond 'maybe' with that particular idea.
For a junior programmer, the benefit of such constraints is greater because they do need the compiler to tell them no. The benefit is somewhat reduced for senior programmers because experience leads them to naturally avoid things that a Haskell compiler would have yelled at them for.
I wrote Haskell for 10 years or so, both FOSS and professionally. I've "been around the block" so to speak and consider myself to have a decent view of the landscape. Overall, Rust lets me code in the style I want while being very resource efficient. I write Rust professionally.
I mean, if I left Haskell, I think the main reason would be to to shed the load of thinking simultaneously about laziness and how it interacts with optimization. OK, almost any non-Haskell language gets you out of that. But choosing to leave Haskell specifically for Rust would be about efficiency first and foremost, especially improving on the size and complexity of the RTS (and its limited platform support). I could also see wanting the concurrent programming benefits of Rust's ownership system. And it's nice to be able to write embedded or kernel code. And there's a bandwagon to jump on.
Lisp, on the other hand, doesn't really seem like an improvement over Haskell in any of those ways. It solves different problems. Lisp feels like it's on the "opposite side" of Haskell from Rust. So why did you "reverse" and try Lisp to begin with?
I agree that Rust is ugly, by the way. Honestly I think it started with keeping the C syntax and went from there.
I transitioned from Haskell to Rust to capture efficiency and small binaries. Since I'm in the game of shipping CLI tools, this was important for me.
You're right though that Lisp is on the other end of that; we're back to bigger runtimes with no tree-shaking, since that would hinder debugging. For now I'm experimenting with the Interactive Programming paradigm because the debugging story is just too good. For long-lived programs, this may be the way to go.
Rust code can be made nice to look at it, but it isn't the default nor the trend.
What I would like to do is to use Lisp to solve complex problems. The problem is that abstractions are not often efficient.
For example, consider computing the calculus product rule. An S-expression abstracts the function: (lambda (u v) (* u v). With metaprogramming, this becomes a new S-expression (lambda (u v) (+ (* (d u) v) (* u (d v)))), where (d u) and (d v) further compute the derivative. To optimize this further, you need implementation dependent code and replace abstractions of S-expressions with native machine code. This is where I feel that the troubles with Lisp start.
This can be done by a program called a compiler. There are many compilers for Lisp.
(In other words, the same argument applies to every language ever which doesn't exactly correspond to machine code.)
I probably haven't actively written anything in Scheme in a decade, but it later inspired me to learn Haskell to a decent level of depth, just to see what the "other" functional languages feel like. So much the same, yet so much different.
One thing I noticed is that those who hadn’t coded through their teenage years found scheme much easier to learn than other languages we were being taught in the first year. I think that was because scheme felt more like mathematics and everyone had done A Level maths (in the UK), many to an advanced level.
Immutability and functions make a lot more sense than weird variables that keep changing and - what the heck are classes? We just unlearn this mathematical thinking when we start hacking Python etc.
It's still one of my favorite problem solving languages, but far from trivial to fix from the inside.
Tried writing my own readers that compile to Common Lisp.
Then I ended up on a side track to write my own interpreters; mostly Forths at first, then gradually more Lisp-like and practical creations.
My latest attempt may be found here:
Make your own lisp. I chose that route even though I had never written lisp code before. Finally understood its elegance.
I'm not sure exactly what feature the OP was referring to. It sounds like they don't think you can get a REPL for an executing Fennel process? You can. If you are only using it for AOT Fennel->Lua you can't, you have to include its runtime.
https://git.sr.ht/~technomancy/fennel/tree/1.4.0/item/change...
I remember the Lua C API exposes a lot of information, but I didn’t think it was accessible from scripts. Of course, it was a long time ago and I could have easily missed something at the time. Happy to be corrected!
Edit: you might also run into difficulties trying to redefine non-global functions to add the call to “repl”.
Also, what was the objective behind shaping the text as a Q&A-style interview when it's only a personal blog?
The site is designs as it is because I'm not a frontend guy xD
I love Lisp's ideas but I find all modern implementations terrible. I would go so far as to say Lisp aren't popular because what we have today is not good enough. Common Lisp is the most advanced one, but it is like C++: it does everything and the kitchen sink. Design by committee. You want functional programming AND imperative? You want documentation that reads like an IBM mainframe manual? You want a standard library with names as cryptic as ANSI C? We got all that. But frankly, it is the one to build serious production software with.
Racket is the best for a beginner, but it keeps having that academic, "we made it for the kids" feel of being easy to understand but not very pragmatic for a seasoned programmer. And it is one of a kind, so a good Racket developer might never know Lisp itself.
Schemes are fun, but the only standard everyone agrees to relegates to small embeddable languages almost like Lua. They are not made to live on their own almost. Since the standard is very simple, there are half a million implementations that are not very practical. But of these, there is Guile, which has a pretty decent documentation AND standard library (even a built in PEG parser!), actively developed and it is my opinion that is it very underrated and the only Lisp worth my time these days.
To be honest, the most popular Lisp in the world is Emacs Lisp bar none. How's that Guile port coming along anyway?
My (least) favorite example:
(nth needle haystack)
(aref haystack needle)
And of course CL tooling is just weird if you aren't used to it.
Quicklisp is amazing! But it has no CLI and doesn't use HTTPS for package downloads.
Swank and Slynk is amazing! But clients other than Slime and Sly are second-class citizens.
SBCL can generate a compiled binary, cool! But the routine is called "save-lisp-and-die".
ASDF expects you to either put all of your Common Lisp projects in a single directory, or hard-code project locations in a config file. Does Go still do that too?
Ancient alien technology for sure, but at least moderately damaged upon crash-landing. Ironically, the only piece of CL tooling that feels mostly not-weird is called Roswell.
Of course all of this stuff is completely free-as-in-beer and developed almost entirely by volunteers in their spare time. Hard to criticize: they built it, and I didn't. But there is definitely a cumulative oddball feeling to it.
https for QL: https://github.com/rudolfochrist/ql-https
another package manager: https://github.com/ocicl/ocicl ("built on the world of containers")
Can't really refute the weird feeling. But there's often a reason! (like, it makes totally more sense to install a library from within the Lisp REPL and not from the terminal, we can use it right away)
(let ((my-map `#M((foo ,foo) (bar ,bar) (baz ,baz))))
(gethash 'foo my-map))
Constructor syntax is nice-to-have, but I don't think the lack of it is evidence of alien technology. Java has the same problem, for example, and Java is about as non-alien as you can get. Overall, hash maps are quite well done in CL, and good surface syntax is easily found in popular libraries, if that's a pain-point (serapeum, etc.).
I'm happy to help you find sb-unicode. :) I rarely use CL these days, but when I do, I'm so grateful to the SBCL team for their amazing and ongoing work.
C? :)
But for a language whose standard hasn’t been updated in roughly 30 years, it holds up impressively well!
...And?
I'm a fairly experienced professional programmer with a lot of systems experience, and I find Racket an extremely well thought out, competently implemented and high performance (particularly with the switch to Chez Scheme) language.
I wrote a bit about it at https://nikhilism.com/post/2023/racket-beyond-languages/ and Bogdan Popa and Alex Hirsanyi have done amazing stuff with the language https://defn.io/ https://alex-hhh.github.io/index.html including a sophisticated web framework, a Kafka GUI client and so on.
I think the only deficiency for a "pragmatic for a seasoned programmer" (beyond the standard "no FAANG is sponsoring it" related lack of resources and presence in popular discussion) is the lack of good editor integration beyond emacs. That said, I've found DrRacket perfectly reasonable for my uses, although clunky. I'm not saying it can replace something like Python, but that is more because of the smaller ecosystem and contributor base than anything wrong with the language itself.
As someone trying to write definitively about cold hard reasons Racket remains in relative obscurity, I would be interested to know specifics you can provide for this view. Is it just academia vibes? Or are there specific aspects of Racket that in your view repel “seasoned programmers”?
I notice you praise Guile for having decent documentation and standard library. How do you believe Racket’s documentation and standard library compare to Guile’s?
Like, there's thousands of libraries and modules in Racket, sometimes competing with each other, that feel like they have been developed for a school project and kept around. And it is no secret that Matthew Flatt, one of the lead developers, is actually a professor at Utah University.
In my experience, it is pretty easy to tell when a language and ecosystem is developed in academia, or developed by software engineers by trade. They have two very different goals and approaches to the same problem. The former might focus on educational purpose for newbies, the latter on shipping production-ready software for professionals.
(In my humble opinion, this is one of the reasons Smalltalk never went very far in the real world. Too much focus on the educational aspect of it.)
"A Shipping Industry Case Study"
https://seaside.gemtalksystems.com/docs/OOCL_SuccessStory.pd...
How do you know what corporations used to develop custom business systems?
This is anecdotal and over a decade ago, but quite specific: my first exposure to Racket was with a version of the language (I think it was some pragma-type thing like `#lang`, but it's been a while) allegedly intended for CS classes, and included something along the lines of [exact spelling and phrasing almost certainly differ]:
> (cons 'foo 'bar)
error: cons: 'bar is not a list
This is the sort of "Dangling by a Trivial Feature"[0] thing that there's just no excuse for, like trying out a new text editor and discoving that pressing backspace inserts the text "^H", except it was clearly deliberately aimed at people who didn't know any better and would 'hopefully' not realise there was a problem, rather than just being a local idiosyncracy[1], because cons did work correctly under other dialect settings.
Actively trying to fuck over newcomers who are only just learning LISP (or programming in general) deeply offends me.
0: https://prog21.dadgum.com/160.html
1: In which case I still wouldn't use Racket myself, but I wouldn't be actively opposed to anyone else doing so.
Analogy: On a calculator, what should the result of sqrt(-4) be?
Depends: If the user is a third grader, then maybe "error, can't take square root of -4". If the user knows complex numbers, then "2i".
The teaching languages does just that. They are linked to the book, which tells you when to "level up".
Conflating the teaching languages and Racket is not helpful.
That has nothing to do whether the user knows better than to trust such a answer; it's because they (likely implicitly, which is admittedly unfortunate) asked for a real-valued result, which indeed can't be done. Eg, in python:
>>> import math,cmath
>>> math.sqrt(-4)
ValueError: math domain error
>>> cmath.sqrt(-4)
2jA language called ISLisp (ISO Standard Lisp) also does this nasty thing.
> just being a local idiosyncracy
albeit a stupid one; whereas Racket actively targets beginners in particular.
Also, this is ISO we're talking about here; we probably shouldn't expect anything technically competent from them that isn't just a rubber-stamped version of someone else's specification, and they didn't call it "ISO Standard Common Lisp" or "ISO Standard Scheme", presumably for good reason.
SRFIs suck really badly. The language isn't useful without them, but every implementation uses a different subset. You have to find out which SRFI does what you want (there may be more than one) then find out what number it is then look up the SRFI itself because the spec is pretty much the only documentation you're going to get.
This entire situation sucks for experienced devs and simply kills most beginners before they even do anything.
Things were supposed to get better with R7RS. It was supposed to have a tiny core language to keep the R5RS crowd happy, but add the R6RS features in the large edition that are needed to get real work done.
R7RS-small released in 2013. That's TEN YEARS ago and it STILL doesn't have a standardized library because apparently nobody wants to work on R7RS-large.
The whole thing is an unusable mess for no good reason and it's killing the language.
https://github.com/johnwcowan/r7rs-work/blob/master/WG2Docke...
I find Racket to be more pragmatic than Java, Python, and JavaScript. So much so that I trade options with it. Edit: "pragmatic" not for all things, but for many things, including GUIs and charts. Second edit: whoops. I see I've already pointed out this Racket program to you.
Have you considered using Typed Racket for it? If yes, then how do you see the tradeoffs?
https://github.com/evdubs/interactive-brokers-api/blob/maste...
This file just includes request message definitions and to-string (->string) implementations.
The overall program's performance is adequate for me, so if the contract system is causing overhead, I don't particularly care about it. Maybe that's a scalability concern for more performance-demanding programs which could benefit from Typed Racket.
It's just not. It was someone's GSoC project, and they basically stopped working on it in 2015 and nobody picked up the slack since: https://git.hcoop.net/?p=bpt/emacs.git
We only got Nativecomp in elisp because a very talented and persistent compiler engineer from ARM saw through it. With the basics done, I think they want to tackle doing more aggressive optimisations with libgccjit. But best not to take things for granted when bus factor is 1.
https://github.com/lem-project/lem (as a realistic climacs) is probably our only hope for a "new Emacs".
I know why you got that reasonable impression, but there's more to it, which can make it much more interesting to practitioners...
The gang-of-professors reasons for Racket are for research platform and education platform.
However, at the same time, Racket (then called PLT Scheme) attracted a disproportionate share of a user community of high-powered programmers and software engineers, like you also see with Common Lisp and some other languages. And one of the professors, Matthew Flatt, happens to be a great systems programmer, with good software engineering sensibility.
There's some really solid stuff in Racket, and I've used it on important systems with hyper-productive teams (you'd think they had a hundred engineers, when it was only a few).
The impression you get might be for two reasons:
1. The gang-of-professors decides most of the customer-facing image for Racket, in various ways, such as writing some introductory books, promoting their zero-prior-experience-student-oriented IDE as the Racket IDE, odd language on the Web site, etc.
2. At times they've also made direction decisions that suit either their education&research goals or, secondarily, their idea of what professional practice wants. On the latter, of course, people who've been professors for decades aren't going to have all the insights of the best people who've been practicing in industry that same period.
If you want to approach Racket as a serious and skilled practitioner, one way is to focus on the Guide and Reference books (I wish these were consolidated), and expect that you'll have to creatively build much of the ecosystem bits you need from scratch. That can actually be a great situation to be in, if you're up for it.
The governance model, last I checked, is a benevolent dictatorship by the gang-of-professors, and the industry practitioner user community is currently small, so just be aware of that going in. But, there's an open ecosystem for packages, more empowering than most languages, due to all the language-extension mechanisms of Racket, so you can "build out" Racket in a decentralized way, to some extent. If you do that, you'll want to keep on top of what the gang-of-professors are doing, and give them a heads-up on things you're doing, to minimize unpleasant conflicts.
I'm not going to make a sales pitch for it; just wanted to add some info for serious practitioners who're already interested, on what to expect, and how to approach it.
I'll just add that the ideas and many libraries around Racket are super cool. There's really a bunch of documentation and tutorials for a lot of things, it's fun.
Seems like F# is trying to solve the same problems as LISP. Lot of same features.
Edit: Did like article. Is serious question
LISP-> Clojure-> F#.
There are similarities.
Edit2:
Guess it is a stretch. But not insultingly so.
Have seen articles comparing Lisp macros to F# Type Providers.